OK, I know it’s been awhile. I’ve been a little preoccupied with work, and studying, and teaching as well. Give me a little break! Currently, I’m focused on two main concepts:
– The idea that there is representational entropy in hidden states (not cross-entropy loss, which is basically saying “How surprised was the model when the real answer was revealed, as it makes predictions).
– Building out a security-first agentic harness for local models.

Why do I care about these things? I care about representational entropy in hidden states because it’s sort of unsolved? Estimating differential entropy in high-dimensional continuous spaces is pretty fucking hard. This is because with all of the current literature, and layering into superposition (features packed non-orthogonally into shared dimensions) no one can really say with ultra ego and perfection how much information a transformers residual stream carries at layer N, let alon how it changes as it trains. There really isn’t any tooling that exists for understanding, measuring, or capturing this idea. And it’s pretty important. However, it’s a little out of my league. I originally was focused on taking a look at predictive/policy entropy dnamics during RL post-training (When labs do RL on verfiable rewards to build reasoning models, the policys (the reward policy) output tends to collapse. We do things like high-entropy tokens, KL anchroing etc. But we don’t know WHY collapse happens and which token’s entropy matters. I want to know what mechanism controls entropy during training. If I can know, then I can build the mechanism to control entropy. And if I can control the entropy, I can regulate it (like how we normally regulate learning rates). I don’t want it to be like a bonus. A “ether” type variable. If we can stop RL runs from leading to explorations dying, it would modify the performance vs compute curve, and extend it. I can take the same resources, and get more capabilities. If I can get more capabilities, we can stop metering the usage of “utility” based LLM usage, and everyone has the ability to train their own models, on their own hardware, at home, with the same reasoning capabilities (with some hardware innovations in the middle). I think with what I’ve reviewed in regards to representational entropy in hidden states, and predictive/policy entropy dynamics during RL post-training, I can bridge the two maybe? Can I answer:

How does entropy in internal representations product the output-entropy we see during training, with the focus being during RL entropy collapse that basically every (I assume) frontier lab is battling.

This would be big. If I can find a way to instrument representational entropy across checkpoints during RL tuning, how outline how internal information structure changes as policy entropy collapses, it’d be a big fucking deal. However, It takes a HUGE amount of review of everything, going back to first principles in some areas, and having the funding to access computer that would allow RLVR on 7B-class open models.

The second thing I’ve been working on is more close to home, and something a little more my speed (SWE can be done at home, insane reasearch takes resources!). I want to make an open source agentic harness for locally hosted models (WE HAVE OPEN AI AT HOME!) that is focused on the fact that we should have zero trust when doing things that involve our data, tools, money etc. I want to be able to have each tool call the agent makes to be passed through capability-based permissions, use more sandboxes, and create a way to block out the agent from tampering their audit log (which like…. they seem to keep doing). This way, anyone, anywhere, using like LM studio, or a locally hosted model can start spinning up as many agents as they want, without worrying about anything.

I know this seems like a lot, but when I’m not working (and hanging out with my super amazingawesomefantastic wife and kids, it’s basically all I’ve been doing besides reading and gaming).

I GOT SHIT TO DO.

Until next time, n00bs
-Haxk