A benchmark for closed-loop evaluation of embedded LLM agents for software development is presented, with results showing emerging capabilities of local models.