Connection
Base URL, credentials and a reachable server.
AI coding workspace
Machine-learning data splits
Model evaluation starts with the data boundary. Model Ledger helps you examine split logic and preprocessing code before drawing conclusions from a result.
● ● ●Model Ledger●
A concrete starting point · Machine-learning data splits ×
train_groups, test_groups = split_groups(group_ids)
assert not set(train_groups) & set(test_groups)
train = rows_in_groups(data, train_groups)
test = rows_in_groups(data, test_groups)✳Split machine-learning data by the entity that could leak across sets.
✓ ChangesReview before applying ↗
Illustrative workspace
Project
Keep each group on one side and fit preprocessing on training data before applying it to held-out rows.
Review before applying
Compare actual behavior with the expected result. If a check fails, investigate it before applying more changes.
A concrete starting point
train_groups, test_groups = split_groups(group_ids)
assert not set(train_groups) & set(test_groups)
train = rows_in_groups(data, train_groups)
test = rows_in_groups(data, test_groups)Give data preparation changes an explicit review boundary
An evaluation result can look reassuring even when the code preparing its data has crossed an important boundary. Model Ledger brings that implementation into a Windows AI coding workspace. Open the project and discuss the preparation path with a connected model, using the files and assumptions behind the result as the starting point.
A useful example is a dataset containing several rows for each subject or source. Ask the model to inspect where groups are assigned to training and test data, then review whether later preprocessing preserves that separation. This is an example of project work to investigate, not a built-in leakage detector or a certification of your evaluation.
Explain which grouping rule the application needs before requesting changes. Review the proposed split code together with its callers and saved outputs, then run checks that compare group membership across partitions. Record what the check covers; a disjoint set of identifiers does not by itself establish that every feature was prepared without access to held-out data.
Model Ledger supports configured OpenAI-compatible or Anthropic endpoints. An abliterated AI model may be selected when your service offers one, but its capabilities and restrictions remain dependent on the model and server. Keep credentials out of sample data and verify the proposed pipeline changes in your own environment.
Give data preparation changes an explicit review boundary — questions before editing
What context should accompany a split-code review?
Provide the grouping key, intended partition rules and code that prepares features before and after splitting. Use representative records without personal details. Explain whether multiple rows can originate from the same subject, session or source.
Can a group-overlap check prove the evaluation is sound?
It proves only the membership condition it checks. Preprocessing may still use information across partitions, and the grouping key may omit related records. Review the surrounding pipeline and state those limits when interpreting the result.
Should I ask the model to improve the score first?
Begin by establishing the data contract and a reproducible baseline. A higher score can reflect a changed evaluation boundary rather than a better model. Ask for justified code changes and inspect their effect on the preparation path.
Does Model Ledger train a model automatically?
The workspace supports coding conversations and review of proposed edits. Do not assume it supplies a training service or dataset. Your runtime, data and configured endpoint determine the work you can perform and the checks you run.
Model Ledger · Updated 2026-10-25
