Yeah - so i haz opinions :) Firstly I'm building dot files to describe my. pipelines, but i'm using the db to store the instantiation of the runs. Gives me ACID semantics for durable execution checking. but the ability for large numbers of agents to read and write in real time from a single source without all of the overhead of git pull/push workflows.
Beads is cool. I don't immediately see the benefits for my use case of using a git repo for the file tracking when compared to a db.
Any reasons you find that better than a db for keeping track of task progress across potentially hundreds of simultaneous agents working across thousands of runs at any given time?
Yeah, ultimately both Beads and SQLite Sync have conflict resolution built in and are backed by centralized databases that can allow many clients to connect, synchronize, write/read/lock at the row level, and merge in real time. That's why I asked if you'd experimented with them as solutions – because it seems like they're accomplishing the same objectives as your Postgres layer, but perhaps with a little more of the "cherry on top" from the get-go.
Makes sense. I definitely need to do a deeper dive. I'm working with my agentic team to review all of the popular tools out there right now, so I will do a deeper dive into them to see if they make sense.
I already have a Postgres system of records set up, so I guess I just hadn't seen what I would win by moving the task communication out of the database and into a version-controlled file that needs to be pulled and pushed. Let me dig in deeper before I say something silly and uninformed 😊
I like your approach - it's definitely more sophisticated than my current system(s). Mine are all per-project / client at this point, which means that each is a bit different which definitely isn't cognitively optimal.
I'm tempted to point an agent at this post and have it build an engineering org like you described for me to play with. Do you have a more detailed description to short-circuit some of the gaps that'll come up?
Great piece. Curious if you've experimented with something like https://github.com/steveyegge/beads or https://github.com/sqliteai/sqlite-sync to solve this problem?
Yeah - so i haz opinions :) Firstly I'm building dot files to describe my. pipelines, but i'm using the db to store the instantiation of the runs. Gives me ACID semantics for durable execution checking. but the ability for large numbers of agents to read and write in real time from a single source without all of the overhead of git pull/push workflows.
Beads is cool. I don't immediately see the benefits for my use case of using a git repo for the file tracking when compared to a db.
Any reasons you find that better than a db for keeping track of task progress across potentially hundreds of simultaneous agents working across thousands of runs at any given time?
Yeah, ultimately both Beads and SQLite Sync have conflict resolution built in and are backed by centralized databases that can allow many clients to connect, synchronize, write/read/lock at the row level, and merge in real time. That's why I asked if you'd experimented with them as solutions – because it seems like they're accomplishing the same objectives as your Postgres layer, but perhaps with a little more of the "cherry on top" from the get-go.
Makes sense. I definitely need to do a deeper dive. I'm working with my agentic team to review all of the popular tools out there right now, so I will do a deeper dive into them to see if they make sense.
I already have a Postgres system of records set up, so I guess I just hadn't seen what I would win by moving the task communication out of the database and into a version-controlled file that needs to be pulled and pushed. Let me dig in deeper before I say something silly and uninformed 😊
I like your approach - it's definitely more sophisticated than my current system(s). Mine are all per-project / client at this point, which means that each is a bit different which definitely isn't cognitively optimal.
I'm tempted to point an agent at this post and have it build an engineering org like you described for me to play with. Do you have a more detailed description to short-circuit some of the gaps that'll come up?
Working on a post and some prompts!