Beyond Fine-tuning

Data ingestion in language models shouldn't be a "clean slate" process. In the Continual Learning approach we are developing with Xctopus, each new learning phase doesn't start from scratch or re-ingest the entire history, but instead operates on informational deltas, assuming that prior knowledge has already been internalized.