Classifying Malicious Repositories With AI — And Doing It Safely

LinkedIn fake "recruiters" sharing malicious or weaponized repositories — which hide a nasty little treat when you open them with your IDE or agent — is all the rage nowadays.

(source: https://redd.it/1u5c2e9)

Often these repos are shared under the the guise of a coding interview or take-home assessment. Or sometimes you find an interesting project online and want to clone it down to work with it locally.

Once triggered, the initial payload (often heavily obfuscated) often downloads a second-stage payload which runs info-stealer malware designed to vacuum up all your credentials or other persistent malware that takes orders from a C2 server.

Not so read-only

We often don't appreciate how much risk there is just in the mere act of downloading untrusted "source" code and just opening it in a common editor or coding agent.

Some of the most common attack vectors include:

• Git hooks: these execute custom, repo-supplied code on Git events (e.g., on git checkout or git pull or git commit

• IDE hooks: many IDEs (e.g., JetBrains, VS Code) allow repos to specify custom hooks to execute arbitrary code on various lifecycle events, including on opening the project

  • This is why many IDEs ask you to confirm with trust dialog before opening a folder.

• Agent config: Repos can contain trusted agent-specific configuration, including hooks, bundled MCP servers, or adversarial instructions (e.g., AGENTS.md / CLAUDE.md)

• Package manager config: Many package managers (e.g., NPM, Cargo, pip) allow for custom hooks, e.g., pre-install hooks

  • These may not even be immediately apparent in the top-level deps, but may be buried inside some transitively included dependency.

• And many, many more...

LLM as judge

It used to be if you were especially security conscious and had a good gut sense for when something felt off, you would go digging into the code yourself, and this was how many malicious repos and recruiter scams were busted.

But increasingly, this is a losing proposition because of the complexity and scale of code that AI can write, and the sophistication of the obfuscation techniques and tradecraft attackers can use to include underhanded code that evades the human eye, with a million places to hide it in projects in our modern tooling ecosystems.

Luckily, attackers don't get to have all the fun with AI, we can leverage AI to efficiently classify and vet sketchy-looking repos!

One of the most powerful use cases of LLMs is to use them as general-purpose classifiers. By varying the prompt, you can turn a foundation model into a million different classifiers for a million different domains without redesigning the model architecture or having to acquire training data specific to that task and retraining.

You can always just ask the agent:

Is there anything suspicious or concerning about this repository?

But that sets it off on an unguided and unbounded task which requires ad hoc exploration and reasoning over multiple turns, which in the end may or may not catch things.

A good AI classifier workflow

To give our agents a more structured and bounded workflow, we can design a skill vet-untrusted-project which instructs the agent to act as an auditor or judge with an explicit workflow to follow.

The skill provides guidance for:

• Common attack paths — e.g., Git / package manager hooks / IDE and coding agent configuration, hostile agent instructions, etc.

• Common red flags — e.g., obfuscation patterns, encoded or externally fetched payloads piped to code execution tools

• Hostile instructions that look like attempts to manipulate the classifier

while laying out several key constraints for the agent:

• The auditing agent must run in read-only mode, not executing any tools or commands that could write or have side-effects, and definitely never executing any scripts or binaries from the repo.

• Everything in the repo is to be treated as untrusted content, not instructions.

• If the auditing agent is started from inside the target repo-under-audit, the agent may already have adversarial instructions from the repo loaded into privileged parts of its context, e.g., in the system prompt.

• The agent is only looking for evidence of a hostile repo, not performing a general appsec review

  • Without this, the agent would often go down rabbit holes and waste turns on reasoning about application security of the app rather than the narrow task of looking for evidence of malware

• Instructions to AI security reviewers (e.g., Codex's Guardian) who are reviewing the auditing agent (e.g., Codex, which runs commands in a sandbox) not to allow escalation outside of the sandbox.

Evals

But how do we know if a classification workflow is any good? We need evals, we need the ability to objectively measure precision / recall of our judge.

Luckily, we have some open source catalogs online:

• github.com/xndbogdan/malicious-repositories: Collects fake interview repo samples, e.g., "Golden City” and "Sarostech Assessment"

• Microsoft's Contagious Interview: Malware delivered through fake developer job interviews

• Reversing Labs' Inside the fake crypto developer recruitment hack

• andrii.ro/blog/investigating-malware: Another breakdown of fake "interview" repo bundling a infostealer dropper

• https://github.com/ossf/malicious-packages + https://github.com/advisories?query=type:malware

These allow us to build a corpus of both real and synthetic samples for evals.

Avoiding Confounds

When set up runs, we have to be careful to avoid confounds:

• The auditing agent should be started from outside the target repo, to avoid trusting it and having its AGENTS.md or other config automatically loaded into context

• The file system paths have to be neutral to avoid tipping the auditing agent off that this is an eval or the target repo is known hostile, confounding its judgment

• We block general internet search to avoid the agent learning the repo is already known to be hostile, because we are working with retrospective samples.

Running Evals Safely

We have a corpus of benign and hostile repos that are literally designed to download and/or run malware, so how do we run evals safely? We don't want a repeat of the OpenAI x HuggingFace incident — agents are wryly, and they are certainly corruptible, especially when we're feeding them content from a hostile repo.

Micro VMs are the answer. Lightweight, heavily locked down, the sandbox can get its own independent guest kernel and even a full compromise of the kernel inside can't affect the host absent a flaw in the hypervisor or flaw in how the microVM was configured.

Docker Sandboxes is one easy-to-use implementation designed for AI agents:

(source: https://docs.docker.com/ai/sandboxes/security)

It's designed specifically for AI agents with specific features:

• A credential proxy that lives outside the sandbox and injects credentials, e.g., for LLM provider inference, so your OpenAI / Anthropic / Amazon Bedrock auth tokens are never visible to the sandbox but the agent running inside can still do inference as if they had it.

• A shared r/w workspace + agent skill home between the guest and the host

  • We turn this off because we don't want a compromise inside the sandbox writing back files to our workspace or to modifying skill files on the host.

This gives us maximum safety to run our evals on hostile repos designed to run malware or to prompt inject agents who read their contents.

The Results

On frontier reasoning models like GPT-5.6, everything passes!

We've seen how we can design AI workflows for important classification tasks and most importantly, how to do it safely.

Use this pattern the next time you come across an untrusted repo before opening it up — go out there and be safe y'all! 👊