Founding Engineer (AI / ML)

Our Company

Pax Historia is defining a new category of gameplay using the latest advancements in generative AI. Our platform brings together the depth of grand strategy with the creative freedom of a sandbox experience, all fueled by a passionate community that creates and remixes scenarios on our platform.

Our community publishes hundreds of scenarios per day, plays millions of rounds a week, and is growing quickly. In addition, we’re backed by Y Combinator, Pace Capital, and Z Fellows. Your work will immediately ship to a product used by hundreds of thousands of players.

The Role

We’re hiring a founding-level ML systems engineer to work in-person full-time in San Francisco (in Dogpatch). You will report directly to the cofounders.

Our current position

The latest closed source models play our game with reasonably good quality, but they’re incredibly expensive.

Open source models are much more affordable but almost never selected by users as their performance on our platform is poor.

Prompts and harnesses are largely identical between models.

A working internal eval system (with vast rooms for improvement)

What you’ll do:

Build and run the infrastructure needed to rigorously tailor harnesses and prompts to each AI model individually to squeeze out maximum performance.

Train domain-specific models to close or even eliminate the gap between open and closed models in their weight class at playing Pax.

Reduce costs associated with closed source models by optimizing caching strategies.

Further improve performance of closed source models by training tuned endpoints.

Evaluate and improve embedding and reranker performance in the places we use them.

Enable entirely new user experiences based on upcoming world models.

• TLDR: Your work will directly make the game more affordable and more fun
• Resources you’ll have:

Trillions of tokens of prompt and response logs from millions of gameplay trajectories.

Tens of thousands of user preference votes per day (coming soon, pairing algo ideas described <u>here</u>)

Generous access to compute (6 figure budget now, with a pathway to 7 if results are promising)

Points of contacts with many of the teams pushing the envelope of inference at scale (Chutes, OpenRouter, CanopyWave, and more)

How performance will be measured

While we understand results may take months to start seeing, your north star metric will be to improve user-preference win-rates over off-the-shelf options with the same inference budget.

This is an intensive role and you should expect to work around 50-60 hours per week for the first few months; after that, hours may begin to decrease. There is potential for slight flexibility (ie, 1 day/week hybrid) but we have a strong preference for candidates who can commit to in person work.

Culture

Most of our players have discovered Pax Historia organically (friends, youtube reviews) and have stuck around because we truly care about the game we’re building. That’s why we want every one of our employees to care deeply about our product too. History, fantasy, or sci-fi nerds are especially welcome, but if you can articulate why you’d be excited to work on our game, we’d love to hear from you.

Pax Historia is still an extremely small company, so you’ll be working directly with the cofounders and a few other employees. You should be self-driven, a team player, and willing to advocate for your ideas. The cofounders will not be hand-holding: their leadership strategy is to ‘get out of the way’ of employees to let them do their best work.

Finally, flexibility is also very important. Since we are scaling very rapidly and still working with a small team, you should come to work willing to help solve a variety of problems on the fly.

Your Qualifications

Core Competencies

You have shipped ML systems to real users and operated them in production.

You have made explicit cost/quality tradeoffs in deployed systems.

You have debugged and fixed unexpected model fail