Domain Adaptation
Domain adaptation is the process of customizing a generative ai foundation model (FM) that has been trained on a massive amount of public data to increase its knowledge and capabilities for a specific domain or use case
What is continual pre training
Continual pre training refers to the practice of taking a foundation model and progressing the training using large quantities of unstructured data
| Approach | Training Set | Training Set Size | Implementation Complexity | Cost |
|---|---|---|---|---|
| Prompt Engineering | Not Needed | 0 | Low | 0 |
| RAG | Not Needed | 0 | Low-Medium | 0 |
| Supervised Fine Tuning | labelled | can be less with PEFT | Medium - High depending on use case | $100 - 5K |