Where does the human boundary sit in finance work?
My guess right now: the agent does the work, a person checks it and signs off.
I haven’t tested this on real work yet.
The way I’ll measure it is minutes I spend per report on Sabiniana. If the minutes go down and the reports stay right, the agent is doing more of the job.
What I have so far:
- DeltAgent’s eval average went from 49.3 to 92.5 out of 100 over 4 rounds. That’s on made-up companies, so it says little about real work.
- On Sabiniana every report goes through me before it’s sent. No reports have been sent yet.
I’ll update this when Sabiniana has its first practices.