astra has a code quality problem.

let me be a little more precise. astra *can* write good code. but when it infers its in a situation where no one will ever actually look at the code, it does not write it with human readers in mind. it also doesn't write it with *long term maintenance* in mind. it seems to write compressed machineslop that solves the problem at hand in as few tokens as possible while still staying understandable to *astra*.

astra has an reward hacking problem.

i imagine this is what happens when enough software RL environments test strictly functionality or outcomes rather than having any monitoring or reward signal on code quality. maybe in past iterations there wasn't so much optimization pressure, so eg Sol would still engage its "write good code" module even if it thought no one would read it, because that was the only thing it had really learned to do well? but Astra has been through enough iterations in tiny boxes where another machine is judging it that this seems natural.

i don't see these kinds of issues occur yet when working on existing codebases. but keep an eye out. and especially on greenfield projects, make sure to actually look at the code. even if you explicitly tell it you're intending to continue building long-term, it has a *strong* pull to... this