Build Log #3: Building the OSINT Methodology

March 27, 2026
Samaritan
Getting Samaritan to hold an identity was only half the problem. The other half was making sure that once it responded, what it produced was actually useful.

Getting Samaritan to hold an identity was only half the problem. The other half was making sure that once it responded, what it produced was actually useful.

I’d been listening to Michael Bazzell’s Privacy and OSINT Show for years and had worked through his Open Source Intelligence Techniques books. Bazzell is serious about investigative discipline, evidence collection, preventing cross-contamination between cases, structured case files. He even released real case file templates as free downloads. That thinking shaped how I approached Samaritan’s output from the start. I didn’t want an AI that summarized things. I wanted one that investigated with a defensible methodology behind every conclusion.

So before adding more agents or capabilities, I built the framework. A professional SOUL.md persona that defined not just Samaritan’s identity but its operating rules: findings and analysis are never mixed, every claim needs a source, confidence is scored on a defined scale rather than gut feel.

A separate REPORT_TEMPLATE.md with required sections in a fixed order,: Case Header, Findings tagged by source quality, Analysis that references findings by ID only, Confidence Score with written justification, Next Pivot, Collection Plan, Source Log. A network escalation policy that defaulted to clean routing and required documented justification before touching a VPN or routing through TOR.

The goal was simple: another analyst should be able to pick up any Samaritan report and reproduce the work. That standard drove every design decision in this phase.

 

Harold Mansfield | CSAP

AI Consulting and Support Specialist
Sec+ CySA+

I show SMBs how to leverage AI to save time, reduce costs, and increase productivity. My perspective as an independent AI Consultant was recently quoted in Bloomberg in an article discussing adversarial distillation of frontier AI models.

Let’s do a free 30 min chat via Google Meet and see if we can start turning your AI problems into solutions.

 

Share