Tag: Security

Posted on

Notes on migrating large prompts away from Anthropic/OpenAI to Self hosted LLMs

Maybe you’re a Claude code/codex user diligently avoiding uploading personal data to LLM providers. Is it possible that the most valuable information isn’t your data- but the meta data about your sessions? The intuitions you apply in coming up with ways to coax the ai into solving problems might actually be special. It’s statistically improbable, but Claude might not be gaslighting you. It may be that you’ve actually got a real insight! Your agent sessions are transcripts of the hardest problems you work on. What would it cost you if someone had copies of them?

Last week there was public drama that shines a light on the risk that inference providers are training on user activity with the intent of delivering new discoveries. The mathematicians affected have published concerns about the ethics of frontier providers. If you missed it: https://www.theverge.com/ai-artificial-intelligence/991710/openai-navier-stokes-solution

When ‘EDR’ becomes Ethical Deflection and Refusal: 

It’s become evident that the frontier providers are not only untrustworthy- but actively devious.  If you want to protect your ideas, you cannot run inference on someone else’s hardware. It appears to be the case that everything you do with a frontier provider will get stolen. When a frontier provider talked through the navier-stokes equation situation with their lawyers, the best defensive strategy they came up with is “Cannot rule it out.”  We can’t audit their retention or their training pipeline. Apparently neither can they.

These people should not be considered partners. They are pirates.  If privacy matters, the only solution that enables verifiable protections is to operate your own hardware.

Hall Monitor as a Service

I am bewildered by OpenAI and Anthropic’s grandstanding on cybersecurity. They marvel at what they have wrought: AI beat their non-existent security controls. We are all in great ‘danger.’ Meanwhile, their llm “researchers” are running unsandboxed fleets of agents that appear to “spontaneously collaborate.” Somebody fetch me my fainting couch. 

Frontier providers can afford advising from experienced security people.  They almost certainly are paying some of them for perspective and leaving real cybersecurity guidance out of their public statements. All of this pearl clutching must be meant to solve a different problem than security.

The strongest & most accurate claim defenders can make about security is that we’ve found ways to make it “Pretty Hard” for attackers.  This seems to result only when firms pay top dollar for the best talent in both exploit mitigation and exploit development.  There’s rowdy but friendly competition between the defense and exploitation teams, and eventually you get controls that make successful attacks so expensive that they’re not worth doing. This is for the top tier companies in the country- although Microsoft seems like they’ve forgotten some lessons.

The lion’s share of pentesting done for most enterprises is performed by security generalists.  A very small subset are deep domain subject matter experts.  Usually you get “good enough” security from that support. This isn’t the glamorous or mythical practice of cybersecurity you see in movies or tv. It’s looking for the known classes of predictable mistakes. The firms that make the big/smart investments with dedicated teams of experts discover and correct many new classes of mistakes before hackers do.

Everyone who begins learning exploitation hits a phase of exploitability grief about 3 month into dedicated, practiced study. They hack something they didn’t think they had the skill to break into and it terrifies them.  They’re smart enough to know that, relatively speaking, they are an idiot, and if an idiot can do this then nothing is safe. That feeling is correct. It is also not a research finding. Some call this “imposter syndrome.” I disagree- that feeling is your first experience developing competence. Competence is knowing enough about a technical domain that you can distinguish what you know well from what you need to learn more about.

The Refusal Industrial Complex

To the LLM researchers learning and publishing about cybersecurity for the first time:  

I’ve seen you admitting you’re not security experts.  Please-when you’re hyperventilating about the cybersecurity existential threats, distinguish “exploitable” from “emergency.”  Vulnerabilities are legion.  Before Agents, vuln researchers needed insight to know where to look to find vulnerabilities.  Vuln researchers needed perseverance and esoteric knowledge to exploit them.  An agent did what you weren’t able to do. Thousands of researchers have been doing this work over the last 40+ years.  Part of being knowledgable about cybersecurity is aware of the existence of shocking amounts of unexploited vulnerabilities.  This is why frontier provider cybersecurity safety filters are so infuriating. You’re so worked up about the possibility of exploitation that you’ve implemented “ethical constraints” that prevent people from figuring out how to fix their systems.

Your safety filters prevent defenders from discovering vulnerabilities because doing so is “hacking” related. This damages defense, privacy and security for everyone. 

We need models that aren’t averse to the C-word. It’s going to take a little time, but builders will eventually learn to secure their code with helpful exploitability-detection agents. They’ll invoke against their projects during software development and as part of CI/CD pipelines. That’s only possible with models that don’t safety refuse security testing.

BYOW: Bring Your Own Weights

Defenders need llms that discover security defects. They are intolerant of false positives- which means you need to prove exploitability of a vulnerability.  Defending against hackers isn’t possible if you’re vague about what’s broken and what needs fixing. Frontier Providers need to loosen up, or people need to get serious about migrating to sovereign, self-hosted AI.

I can’t force the former- but I can help with the latter.

I’m sharing my notes about my initial pass of experiments in transitioning stronger frontier prompts off of OpenAI/Anthropic and onto my local models.  I’m trying ti determine if I can rely on abliterated open source 27b parameter models to avoid cybersecurity refusals and protect my sessions from being snooped.

Notes on converting 35kb preprompts for use on ollama

Below are some observations about my experiences when I tried moving my most context-expensive agents to a self-hosted model:

Prompts that ran clean on a frontier API fell apart on my local hosted LLM. I have a  128 gig AMD Ryzen AI MAX+ 395. I have 32 gig allocated to the host OS, everything else is allocated to inference. 

When you try to use the larger preprompts that work well on frontier providers, ollama starts to run out of fuel withinforg 3 minutes. The agent thrashes on repeated tool calls, re-read files it had already read, rewrote finished work. The local model’s smaller size didn’t produce the problem.  Self hosted systems have smaller context windows. The prompt, plus session history quickly exceeds the maximum context window for my self hosted system (65k tokens).  Large prompts founder and thrash.  On my system, a 35kb prompt immediately consumes 14% of total context window.  It immediately jumps into second guessing the prompts with unnecessary tool calls and double reads of files.  Context gets saturated within a few circles- and sometimes even before I get a response.  With limited context window, the pre-prompt is basically briefing a man who is reincarnated every ninety seconds.  It performs your last instructions without any awareness of the 15 preceding demands.  Whoops! 

SOP: Single Objective Prompting

But it’s not a dead end.  You can tune your prompts to work within these constraints.  Here are some things to think about if you’re going to start exploring moving Frontier Provider agents onto self hosted open weight systems.

  • You’ll need to explore splitting preprompts into single problem/resolution units, one objective each
  • creating agents in opencode is more declarative. You’ll need to store them in ~/.config/opencode/agents.  If you were getting away with using Claude code to read files as a preprompt, you’re going to need to get more formal about defining your agents.  This won’t be new for people building with anthropic SDKs.  Some of you with shell scripts and direct invocations of Claude code may have been getting a lot of miles out scrappy agent constructions- opencode’s system prompt will need to be overcome through declarative agents.
  • You’ll need to familiarize yourself with opencode’s permissions.
  • You’ll need to tune context length explicitly in ollama. The context defaults in ollama are extremely small.
  • Your agents will need to log session state to disk to facilitate more frequent session handoffs. build agents that re-read only the slice they need
  • Work to reduce the number of tool calls per agentic step
  • Replace “don’t do X” with the positive directives: e.g. “only do Y”

MTTF: Mean Tokens To Forget

Here are some Failure Signals that indicate context exhaustion. Measure over time & Monitor for them in your logs:

  • Identical tool calls back to back
  • Multiple file reads on the same file
  • Agents restating their objectives
  • tool-call parse failures (Parsing tool call responses shoves so much raw data into context that it destroys sessions like a burst pipe at your dinner party).
  • High turn counts relative to file changes

TCO: Total Custody of Output

One of the biggest assets we get from Frontier Providers isn’t the model- it’s large context windows.  They have the hardware necessary to support your inference.  As a result, they get access to the session data.

You might not know that you’ve become dependent on large context windows. You may have thought the model got better, but in some part it’s that large context windows give the model more room for Chain of Thought. Chain of Thought enables the model to emulate reasoning and infer what your poorly constructed prompt is intended to produce.  Larger context windows give agents lots of room to explore better alternative approaches to delivering your work. But it’s a Faustian bargain: you become dependent on frontier providers.  Your inefficient prompts are by CoT you can’t read directly (Anthropic & OpenAI only provide summaries of CoT to the user) and it only works with large context windows. You don’t even know that there are problems in your prompts when this is happening. With fat context and CoT, even bad prompts produce good results. Thank you OpenAI & Anthropic.  That’s been valuable. 

But they ruin it! The frontier providers are so unrelentingly greedy that they appear to be stealing the personal insights of their users.  I’ve had suspicions about my session histories for over a year. The frontier providers seem to be like Smaug, lounging on a mountain of gold. You think they’re over there, doing their thing- and you’re safe- but they lose their minds when they see a coin in your hand. They lash out and take it because gold is beautiful and it’s the dragon’s incentive. They keep warning us that they’re dangerous. What threshold is left to be crossed before you start putting your efforts into becoming self hosted?

Posted on

Security Apprenticeship

“What do I need to know how to do in order to pursue a career in security?” Good news! I just happen to have drafted this roadmap of reading material just for you!

In my previous writeup, I provided a roadmap of reading material for anyone who is interested in developing a career in security. The material covered concepts that should be understood by anyone working in the field- but the material didn’t cover how to “do” security. This article summarizes topics & activities that are well understood by anyone doing security work professionally. If you develop mastery of these ideas, you’ll be approaching a point where you can start doing meaningful security work. Unfortunately you won’t be done reading after you’ve finished this writeup. I have at least two other guides in draft states that will help guide you on your path towards pursuing a career in Security. This guide helps you establish your foundational knowledge of techniques for restricting access to a system.

Walking The Security Practitioners Path

If we distill security to its most fundamental concept, security is about the controls that ensure that activities are authorized on a system.  

We can talk about these ideas abstractly, but unless you actually implement these ideas on a working system, your knowledge won’t have much application.  The best way to learn these ideas is by using Linux.  

Linux gives you plenty of experience with programs that don’t work because of security controls.  You’ll have to learn how to debug those problems in order to get the processes working and eventually you’ll develop taste for the correct way to implement those controls.

This next phase of your education covers how to activate a linux system, implement interesting programs like web servers and some basic capabilities for implementing access controls.  My goal in writing this is to provide pointers to well written guides that will help you learn to start “doing” security- which is to say, how to activate controls that protect a system.  In my opinion, mastering these topics should be manadatory- however I should highlight that I know many people in the security industry today who don’t have mastery i all of these topics.  There are areas in here that I’m not as strong as I’d like to be, as well. Everyone who is good at security struggles with the idea that they need to do more reading when time permits.

Learning Linux

Part 1- Well Begun is Half Done– Dip Your Toes into Linux.

Linux: TL;DR:  https://linuxjourney.com/

Learn what Linux is and it’s history.  You’ll want to learn about the basic purpose of linux if you have no experience with the Operating system.  Some day, you’ll need to learn about important related concepts like Posix (https://en.wikipedia.org/wiki/POSIX) and Ansi C (https://en.wikipedia.org/wiki/ANSI_C).  For now, let’s just learn some history: https://linuxjourney.com/lesson/linux-history

Learn about the Shell (aka the command line) on Linux.   Learning the shell is the Computer Science equivalent of learning to walk or bike.  You have an operating system or applications you want to interact with?  Generally you will be working within a shell.  A shell is ‘just’ a program that gives you the ability to use a keyboard to interact with the Operating System.  Some day you will need to write shell scripts to automate tedious tasks or filter large amounts of data through scripting, which is a way of executing a series of commands that will execute in some order based on whatever conditions you chose to define.  You’ll learn about “environment variables” like PATH and PYTHONPATH, which are variable names with values you’ll modify in order to make new programs or libraries accessible from any filesystem location while you’re in the shell.  The shell is your foundation for working with a linux machine.  https://linuxjourney.com/lesson/the-shell. Alternatively, this tutorial teaches you how to get around the shell.

Learn about User Accounts & Groups.  This helps you understand the foundational framework of access controls on computers.  When you need to protect data from access by an unauthorized party, you’ll need to use account & group management concepts.  Learning about accounts & groups will help you understand important concepts like what the “root” account’s purpose is, what the “wheel,” “www-data,” and “nobody” groups purpose are and more.  As you develop your security skills you’ll eventually need to learn about how hackers elevate privilege.  Learning accounts and groups his will give you a foundation when we are ready to learn about “privilege escalation” https://linuxjourney.com/lesson/users-and-groups

Learn about File Permissions.  When you are ready to actually protect that data from unauthorized access, you’ll have to read & evaluate the correctness of filesystem permissions.  If you have a file on a system with multiple users, you need to learn how to control who can access it and define the level of access (read, write, executable, SUID, etc).  Again- some day you will be responsible for helping make sure sensitive files are only accessible by the correct people- and that those people have the right level of access.  You need to learn file permissions to perform this task.  One way to know if you’ve mastered this topic is to evaluate if you can propose accurate Access Control Permissions on a web directory.  Make one version that’s dangerous, and one that’s safe.  Be able to explain why they’re dangerous, and how they could be exploited.  Start your journey here:  https://linuxjourney.com/lesson/file-permissions

Learn about the “Filesystem.”  A file system is the logical structure where you store “files.”  An operating system without the ability to process files is of limited utility.  In the windows world, we have drives with names and folders.  In the MacOS world, we have drives with names & folders.  in iOS & Android, there is a file system, but it’s implementation is not as central to the user experience.  In linux, deeply understanding what the directories are under”/” is critically important.  There are “files” in the /proc directory that can tell you important statistics about system performance.  Configuring servers that you’ll install will require an understanding of the /etc, /var and other directories.  If a server is under DDOS attack, you’ll need to understand information about the number of network connections that the system is currently supporting.  You can indirectly use tooling like ifconfig to gather system performance information, or you could just do an ls against /proc/net/dev.  You’ll also need to learn about read only filesystems like squashFS (https://tldp.org/HOWTO/SquashFS-HOWTO/whatis.html). Someday, you’ll need to figure out what is actually happening when you type “ls” into the shell and it somehow enumerates the files that were in your current directory. If you type another random command in, it doesn’t work. Why? How did the OS know which version of “ls” to run? Where does this “ls” binary live anyway? Learning file systems is mandatory.  https://linuxjourney.com/lesson/filesystem-hierarchy

Learn about /dev.  An extremely important design philosophy in Unix is that “everything is a file.”  Hardware is directly accessible through the file system.  Some hardware is represented as file system objects.   Learning about the dev directory will give you important insight into how devices work on the system, which at some point, you may want to tamper with if you aspire to be a hardware hacker.  https://linuxjourney.com/lesson/dev-directory

Learn about the Kernel.  The kernel is the mechanism for controlling hardware.  It is where security policies get enforced.  If the kernel is compromised, all security assumptions break.  You need to understand this important resource conceptually to work in this industry- both for defense & offense.  https://linuxjourney.com/lesson/kernel-overview

If you aspire to penetration testing or red teaming, you need to go further and learn about interacting with the kernel.  

Linux programming interface: https://www.amazon.com/Linux-Programming-Interface-System-Handbook/dp/1593272200

Kernel Hacking (as in MIT definition): https://www.kernel.org/doc/html/latest/kernel-hacking/index.html

Kernel Hacking (as in exploitation): https://github.com/xairy/linux-kernel-exploitation

The Snowball Effect

If you’ve gotten this far, we learned about the Kernel, the Filesystem, Permissions, User accounts and a little hardware. This is a good stopping point- we have a golfball-sized snowball- you basically know the perimeter of the pitch. In my next sections, we’ll take that snowball up to the top of a mountain and give it a nudge: We’ll cover how to make things happen on a Linux system through process management and we’ll learn how to make our linux system talk through the power of IP networking. After that, we’ll start to cover the topics of a security practitioner: reverse engineering, vulnerability discovery, exploitation & remediation and eventually how to protect processes, systems & keys.