The method, step by step

How a chamber's Record becomes a table, one speaking turn per row. It is the same method used to build the 16 corpora, and anyone can apply it.

Skills and code

A skill is a file of instructions that an AI assistant follows when a task calls for it: what to do, in what order, with which tools, and when to stop. It is the name Claude Code and other assistants give these files.

There are twenty-one Claude Code skills. Each one describes a step: what it reads, what it does, what it writes, and when it must stop.

The repeatable work is done by deterministic code: the same file always gives the same result. The assistant only steps in where judgment is needed, such as a misrecognized speaker or a name that can be read two ways.

And none of those decisions is final without a person. The method is built so that whoever does the research sees what the machine does not.

From the source to the table

Thirteen steps, in this order. Each one leaves its result in a folder of its own, so you can go back to any point without redoing what came before.

Prepare
1. Set up the country. It reads a few Records, works out their format—how dates are written, how a speaker is announced—and asks about what it cannot work out.
  /diaries-bootstrap
From document to text
1. Recognize the text. Only for scanned PDFs. A model that runs on your computer reads each page; nothing is sent anywhere.
  /diaries-ocr
2. Extract. It gathers each session into a faithful text, with its page breaks. It corrects nothing: it is the point you can always go back to.
  /diaries-extract
3. Clean. It removes headers, footers, and page numbers, rejoins words split at the end of a line, and puts each intervention in its own paragraph.
  /diaries-correct
From session to turns
1. Date and describe the session. Date, number, type, legislature, and who chairs, segment by segment, including each handover.
  /diaries-meta
2. Set duplicates aside. A session published twice is set aside, without deleting it, before going on.
  /diaries-dedupe
3. Mark the turns. It marks where each speaking turn begins and who takes it. It is the most delicate step, and the one with the strictest threshold.
  /diaries-tag
4. Turn it into a table. It converts the marked text into rows, one speaking turn per row, with its order in the session.
  /diaries-matrix
From turns to people
1. Build the roster. The list of who held a seat in each legislature, from the official source or from the roll call.
  /diaries-deputies
2. Link. It links each speaker to their deputy. Doubtful cases go through human review, and what is reviewed is kept.
  /diaries-match
3. Complete. It adds to each turn the party, district, and sex of the speaker, according to the term in force that day.
  /diaries-merge
Publish
1. Align. It brings each country's table to the same 16 columns, with identifiers that are stable within an edition.
  /diaries-standardize
2. Document. It writes each country's README, data dictionary, known limitations, and metadata.
  /diaries-document

Order matters. Duplicates are set aside before the turns are marked: otherwise the same session would count twice when learning how each chamber announces its speakers.

Who decides at each step

Each step scores its confidence in its own result, and the score decides what happens next.

It continues (AUTO). Confidence is high: the result is saved and the process goes on.
It is flagged (FLAG). The result is valid but uncertain. The process goes on, and the case is listed for review by hand.
It stops (HALT). Confidence is not enough. Nothing moves until a person steps in.

Marking the turns and linking the speakers have stricter thresholds than the rest: a mistake there attributes words to someone who did not say them.

To control the process

Besides the steps, eight skills can be used at any time.

Status. What is done, what awaits review, and what has stopped, by country and by session.
/diaries-status
Review. It goes through the flagged cases one by one: accept, correct, skip.
/diaries-review
Correction. It applies a fix to one session, to a range of sessions, or to a whole step, and marks what must be redone.
/diaries-feedback
Decision. It logs a methodological decision with its measured evidence. Without evidence, nothing is saved.
/diaries-decide
Validation. It draws a random sample and reviews it, to measure the errors the process does not see.
/diaries-validate
Report. It gathers into one document what was done for a country, with its decisions and its figures.
/diaries-report
Guide. It explains the method to newcomers and suggests where to go next.
/diaries-teach
Improvement. It corrects a skill based on what was observed when using it, and records the change.
/diaries-refine

Every decision, with its evidence

Any decision someone might question later is written down at the time: what was decided, what the alternatives were, and what measurement justifies it.

Superseded decisions are not deleted: they are marked and linked from the one that replaces them. The project's methodology document is generated from that log.

Getting started

Install the skills, open Claude Code in your project folder, and ask for the guide. It will ask what you have and tell you where to go next.

git clone https://github.com/rodrodr/parlaibero-tools.git
cd parlaibero-tools
bash install_skills.sh

Then, inside Claude Code: /diaries-teach for the guide, or /diaries-bootstrap to set up a country.

It requires Claude Code, Python and, only for scanned PDFs, Ollama, which recognizes the text on your computer. The skills are written in Spanish.

The technical documentation, with the confidence formulas and the options of each step, is in the repository (in Spanish).

View the repository

It is the method as it was applied, in its current version. The exact copy of the code that produced each published edition is kept separately.

Skills and code

From the source to the table

Prepare

From document to text

From session to turns

From turns to people

Publish

Who decides at each step

To control the process

Every decision, with its evidence

Getting started