That is the second memo the place I describe my latest experiences on working
small fashions domestically on my developer machine for agentic coding. In
the primary memo, I lined the numerous elements that may affect the viability of that setup — {hardware}, mannequin selection, runtime, harness. Right here I deal with the concrete experiences, the duties I gave the fashions, what occurred, and my last conclusions.
Scope
To recap: my focus is on agentic coding particularly, not simply auto-complete. The machines I used have been an M3 Max with 48GB RAM and an M5 Professional with 64GB RAM.
Viability funnel
Evaluating small fashions is sort of tedious to be sincere: downloading takes some time (on my non-fibre web connection in the course of Berlin), then configuring the brand new mannequin in a harness, utilizing it for duties, decoding the outcomes, …
I considered it as working via a funnel of accelerating viability:
- Does it match into the RAM? If I haven’t got sufficient RAM, I will not even be capable of run
the mannequin, interval. For the final evaluation of broader viability, let’s assume as a
baseline of 48GB accessible RAM. - Does it run at cheap velocity? As soon as a mannequin is working, the primary smoke check is
to see how shortly it may possibly reply to a easy request. - Can it deal with instrument calling? As soon as I noticed that the request velocity was bearable, I
gave it a easy process from a coding harness that concerned studying and altering recordsdata, to
see if it may possibly deal with the instrument calling. - Does it construct functionally appropriate code?
- Can it deal with a continued dialog / extra context? As soon as a easy process labored,
I continued the dialog for longer, to see how a lot forwards and backwards it might soak up
phrases of size of the context. - Can it deal with a bigger or extra complicated process? If a setup survives 1-5, the subsequent stage could be to present it a extra complicated process to resolve and see what occurs.
- Is the code high quality acceptable? What’s the stability like between velocity of coding and assessment effort?
The journey
It has been fairly a curler coaster journey, which is an statement in itself!
- Part 1 – handbook evals: I began with selecting a couple of duties, after which doing them again and again with completely different fashions and configurations, “manually”. I wished to see what it feels like when it comes to the consumer expertise, which is tough to do with an automatic setup.
- Part 2 – automated evals: Nevertheless, after I shared these outcomes with different colleagues in Thoughtworks, I instantly discovered new issues from their enter and experiences. So to retry a few of my duties with completely different configurations, I vibe coded a small automated eval setup in spite of everything, which gave me extra information.
- Part 3 – day-to-day use: In spite of everything of this, I labored with essentially the most promising mannequin (Qwen3.6 35B MoE) for some time on the duties that stored popping up in my everyday, to see if I might combine it into my workflows.
Duties
Which process to present to the fashions makes a large distinction, and is among the key challenges of analysis. The principle duties I utilized in my extra systematical comparability have been each JavaScript/Typescript, so not loads of tech stack range, maintain that in thoughts.
In a much less systematic manner, I additionally wrote a couple of shell and Python scripts with it, which labored positive.
Because the duties are so vital, I am providing you with some particulars right here for 2 of them, to assist inform your interpretation of my outcomes. The selection of process is finally one of many largest elements that determines viability of small, domestically run fashions – it is all about expectations. It is about complexity of the duty (How good is the mannequin at reasoning?), in regards to the variety of recordsdata we estimate the agent must learn and write (How good is the mannequin at instrument calling? How large is the context window?)
Activity 1: Type and cumulate an current bar chart
I need to change the diagram within the frontend titled, “Messages per
nameless poster”
– It must be “per poster”, not “nameless”
– I would like the bars within the bar chart to be sorted by variety of messages, with highest bar
first on the left
– I would like the x-axis to not present numbers (“#75”), as a substitute I would like it to indicate what
proportion of the general messages a sure bar is at. E.g., if the primary 10 bars add
as much as 240 messages, and the general messages in that point interval are 1000, then I would like
to see 24% on the x-axis on the tenth bar. I would like this proportion to be proven each tenth
bar.
- Wants some code search, albeit trivial (I give away the lede within the immediate, it must search for a selected chart title)
- Wants modifications to 1-2 recordsdata
- The cumulation of values on the x-axis was a factor that fashions most steadily struggled with
- No pre-existing assessments for these recordsdata, no express expectation of assessments so as to add
Part 1: Handbook analysis
I attempted this with Qwen3.6 35B, Gemma 4 31B, Gemma 4 26B and Qwen Coder Subsequent 80B MoE, with each OpenCode and Pi because the harnesses.
- Qwen Coder Subsequent did achieve a functionally appropriate implementation of this inside these 2.5 minutes on my 64GB M5 Professional. Nevertheless, after I added one other message to the dialog after that, the runtime crashed – so it is succesful, however not realistically runnable.
- Qwen3.6 35B and Gemma 4 31B adjusted the sorting with out difficulty, however then it took me quarter-hour to travel with them in regards to the cumulative percentages till that performance labored. Within the spirit of my “funnel” described to start with, at this stage I simply wished to deal with performance, and did not even take a look at the standard of the code – if the fashions can not even give me performance proper, the standard would not matter a lot but.
- Gemma 4 26B was essentially the most profitable right here. It did implement the total set of issues I requested for. However then after I continued the dialog to ask for a refactoring, I bought the “textual content wall of doom”… My understanding is that this may be mitigated with activating “presence penalty”, however I’ve not tried that but.
Part 2: Automated analysis
Frustratingly, the automated setup didn’t affirm the handbook expertise. Gemma 4 26B didn’t ship a functionally appropriate resolution 3/3 instances, wheras Qwen3 35B MoE succeeded 2/2. The failure was all the time that it did not correctly implement the x-axis labels I requested for. They both weren’t displayed in any respect, or incorrect.
Be aware that for the automated analysis, I anticipate the agent to “one-shot” the issue, which isn’t absolutely lifelike. I gave it entry to the browser as the one sensor to self-correct, nevertheless it by no means referred to as it. Increasing sensors and sensor use might result in self-correction that might make this extra viable in actual utilization.
Activity 2: Create a bar chart of nations primarily based on access_log information
I need to add a horizontal bar chart to our visualisation that exhibits which international locations the requests have come from. The title of the chart must be “Nations”. It ought to present the highest 10 international locations, sorted by variety of requests, with the nation with essentially the most requests on the high. All different international locations must be grouped into an “Different” class on the backside of the chart. For entry entries and not using a nation worth, they need to not present up as their very own class, however be lumped into “Different” as nicely.
As a reminder, the entries now appear like this:
{”ts”: “2026-05-24T01:10:41+02:00”, “ip”: “x.x.x.x”, “standing”: 200, “ref”: “https://the-referer.com”, “nation”: “Malta”}
We are going to solely should learn and alter this one file, don’t be concerned about the remainder of the appliance. We’re studying a bunch of *.ndjson recordsdata with entry log entries such because the one given for instance above, and visualising them
@scripts/visualise_access_logs.mjs
- Modifications to precisely one file, a script that generates a HTML web page
- Pre-existing use of D3 for diagrams already within the file
- No check setup for this script, no expectation of assessments being written
Part 1: Handbook analysis
I attempted this with Gemma 4 31B and Qwen 35B, and it was surprisingly tough! I noticed my reminiscence utilization improve fairly a bit, very lengthy reasoning chains adopted by excruciatingly sluggish makes an attempt to lastly edit the file. I attempted some completely different variations, like refactoring into smaller recordsdata with a giant mannequin first. I attempted with each OpenCode and Pi, nevertheless it didn’t make a lot of a distinction. My notes say issues like, “Gave up after 11 minutes”, “Gave up after 12 minutes”, “Stopped after 8 minutes”. No concept what was completely different right here in comparison with the opposite duties, it would not appear that difficult.
As this process required change to precisely one standalone file, I additionally tried to go old skool right here: I began a plain chat with a mannequin in LM Studio, with none harness use, and pasted the total file in there – that gave me a working resolution! Nevertheless, the entire thing took about 6 minutes, and most of that point was spent by the mannequin regurgitating my 450 traces of earlier code, plus the added traces for the chart.
Part 2: Automated analysis
Within the automated setup, I ran this process with Qwen 35B MoE 7 instances, and it didn’t correctly resolve it 5 out of these 7 instances… The difficulty within the failures was all the time that there have been no labels displayed on the bars to depict the international locations.
Right here is the place it will get attention-grabbing although: I then ran the evals once more on the M5 machine with 64GB, and it failed solely as soon as! Which was actually shocking to me, as a result of I used to be anticipating the 2 machines to ship completely different speeds, however I didn’t anticipate this huge distinction in high quality of output with the identical mannequin settings. This stays a thriller to me.
Daily use
Activity traits
Along with these structured comparisons, I additionally used Qwen3.6 35B MoE frequently for everyday duties, each work and private.
- A couple of bash and Python scripts – usually okay
- Including new content material entries to my private web site – good
- Making small, very nicely outlined modifications to current codebases – usually okay
- Constructing a recreation from scratch (planning with Claude Sonnet, then delegating coding execution to the native mannequin) – began nicely, however fell aside for extra complicated logic
These are a few of my reflections on the selection of process, as of now:
- Will it require code analysis, or can the immediate level on the particular recordsdata to alter? The extra discovery the agent has to do, the extra it requires instrument calling, and the extra it fills up the context window, and due to this fact treasured RAM.
- What number of recordsdata would possibly it should edit? Even when we will level the agent at particular recordsdata and skip a few of the code search, the larger these recordsdata are, the extra strain on the context window.
- How particular are the directions? We have all change into used to being much more obscure with our coding agent directions, as the large fashions have gotten so succesful. Prompting brokers with native fashions feels a bit like a mixture of the early auto full days, and my first makes an attempt at utilizing Copilot’s “multi-file edit” mode in November 2024…
- What’s the tech stack? It is onerous to inform from this very small information set, however my impression was that I used to be often way more profitable with the Bash and Python duties than the JavaScript ones. However there might have been different elements at play within the process kind.
Again to fundamentals
In my day by day utilization, I’ve really fairly loved the journey again in time, to weaker capabilities. It felt nearly like a detox. A small mannequin has larger sensitivity to modifications within the setup, so I bought extra alerts about what helps and what would not. And the temptation to simply try and never perceive the outcomes anymore is lots decrease, I went again to reviewing outcomes way more – and I imply that in a great way. My expertise with giving robust fashions numerous autonomy, and surrendering to them in a manner, has usually led to fixed rework and surprises at later levels. So I’ve loved that the weaker fashions make me go slower, and take extra care within the second, as a substitute of deferring that comprehension and care to the long run.
A second perspective
My colleague Jigar Jani works with Qwen 35B MoE (4BIT) frequently on a 48GB Macbook, and does nearly all of his coding with it, on a “actual world” Python and React codebase. He’s constantly enhancing his harness with expertise, and has discovered Graphify and Perceive Something to be significantly helpful to assist the fashions with code search and understanding. Jigar finds it fairly helpful and sees enhancements as he improves the harness, however he additionally stresses that code assessment is tremendous vital.
To sum up…
It has been a irritating expertise with typically complicated outcomes, I due to this fact discover that it is undoubtedly not a plug-and-play kind of expertise but. However, I really feel like I perceive a bit higher which forms of duties have the potential to work with these small fashions at this level, and I’m incorporating Qwen 3.6 in my workflows to additional enhance my instinct when to succeed in for it.
General although, the agentic coding capabilities are undoubtedly very distant from what I’ve now change into used to with larger fashions.
On the time of publishing, my default setup is:
- Mannequin: qwen3.6-35b-a3b, 4BIT quantization (48GB of obtainable RAM are stretching it with that mannequin, I’ve to shut different excessive RAM purposes throughout utilization)
- Reasoning off (LM Studio: Inference > Settings Customized Fields > Allow Pondering > disable)
- Max context window (LM Studio: Load > Context and Offload > Context Size > drag slider to max)
- OpenCode or Pi as coding harnesses
- I attain for it when I’ve small, easy duties, usually pre-planned by a much bigger mannequin