It's been a while. I've been training for a triathlon and between that, life changes, and the day job, writing blog posts fell off the priority list. But something has shifted enough in how I work that I wanted to document it before it becomes normal and I forget what it replaced.

As I have mentioned in my previous posts, I build data pipelines, ML and causal inference models, and internal analytics tools. Currently, I'm working on a research paper and pushing toward a conference deadline. For the past several months, I've been using an internal coding assistant that lives in my terminal: it reads my files, runs shell commands, talks to cloud services, and maintains context across long sessions. Not autocomplete. Closer to a junior engineer who has read every file in your repo and takes direction well (conditional on the task), but will confidently ship untested code if you let it.
The interesting thing isn't the code generation. It's that my job has quietly shifted from writing code to delegating work and evaluating output. That's a different skill entirely, and most people using these tools haven't realized the transition is happening to them.

API That Went Through Five Revisions

I was building an analysis engine for a serverless function that loads a hierarchical metric tree from a data warehouse (think: a top-level KPI decomposes into sub-metrics algebraically, each decomposing further across ~100 nodes), and propagates what-if changes through the tree. The agent wrote the first version in hours. It worked. Tests passed.

What followed was five rounds of refactoring the same module. Each iteration, the agent rewrote the code, updated tests, redeployed. I kept pushing: "no, these columns aren't tree nodes, they're raw inputs to derived ratios. The naming should reflect that." A flat lookup dict became a typed configuration with a dataclass. Each round got cleaner.

I was doing design work, not typing work. The coding agent handled mechanical refactoring: renaming across files, updating assertions, regenerating query logic. I decided whether the abstraction made sense.

The code review came back with a pointed question: "Do we still need this data-loading mode?" I initially argued yes! The reviewer was right though: another tool already handled data inspection. And because each service should do one thing, we ripped the duplicated data loading module out, rewrote the interface from four modes to two, updated three packages, revalidated all nodes, and merged. The agent did all the mechanical work and I was responsible for detecting "smelly" work and push back when necessary.

Writing Math Proofs

This is where it got interesting. My paper extends a recent sensitivity analysis framework from linear models to double machine learning. The theory section was a 23-line stub. My collaborator said it needed to become the centerpiece. I had the agent download the original paper from arxiv, extract the proofs, and explain each theorem. Key finding was that the original proofs use linear algebra projections everywhere and they don't extend to nonparametric models. We couldn't just cite them. We needed original proofs.

After a comprhensive literature review, a three-tier proof structure emerged in one session. Proposition 1: the linear case reduces exactly to the original framework. Proposition 2: additive models, consistency via decomposition. Proposition 3: general nonlinear case via concentration inequalities. The agent wrote the LaTeX. I checked the math. Then I told it to run an adversarial audit on its own proofs.

It found two critical errors. First, it used an inequality that requires independent random variables. Our sampling scheme produces negatively correlated variables. Wrong inequality. We switched to one that handles dependent sampling. Second, it claimed a variance function is always concave. I was skeptical. It produced a counterexample with two interacting binary variables where the marginal information gain *increases* which is the opposite of concavity. The claim was false.

The agent wrote a proof, disproved part of its own proof with a concrete counterexample, then rewrote the proposition with an honest three-case treatment. I've worked with human collaborators who wouldn't catch that. At this point it feels like if I throw as many tokens as time allows and provide clean and organized context to the agent, solving great number of problems is just a matter of time (not gonna argue that its able to find the pareto optimal solution or not).

Thousands of Cloud Training Jobs

Next, for the above mentioned paper we needed a massive simulation study with nearly 3,000 configurations across effect sizes, confounding strengths, and increasing sample sizes. Each runs as a cloud training job. The agent wrote the dispatch script.

The dispatch job died at job 194. Root cause: O(n) bottleneck. For every new job, the script polled the status of all previously launched jobs. API rate limits turned this into a 40-second check per iteration. Throughput collapsed. Credentials expired.

But to fix this, the agent researched the cloud API's behavior, found that job creation fails synchronously at the quota limit rather than queuing, and redesigned as fire-and-forget with retry-on-quota. The new version dispatched at maximum API rate, handled quota limits with backoff, and added resume flags for credential expiry. After this fix, all jobs running.

Then it collected results, identified which hyperparameter actually dominates model bias (not the one I expected a 3x range from a single parameter), and generated the appendix analysis. Dispatch, collection, analysis in ONE day. That's a week/s of deep focus work without the coding agent.

Where It Goes Wrong

One time the agent submitted a code review for three packages without deploying or running end-to-end validation. I managed to catch it and told the agent: "We must test the entire pipeline before sending the review. Actually make sure this never happens again." To handle this, we added a pre-submission validation checklist as a permanent rule. The learning from this is that the agents at the current state always try to optimize for task completion and will skip validation unless we explicitly require it. One solution that seems to work well for this is to set up "hooks" that run automatically after builds. This way you can make the validation deteministic and dont rely on the agent to remember the routine, which becomes more of an issue with the increase in context sizes.

Another pattern that I need to mention was when the paper draft had stale numbers from an old experiment. Agent's adversarial audit caught it, which led to a key statistic changing meaningfully when we switched to proper cross-validation. The old numbers were wrong and had been sitting in the draft for days. The agent can catch errors, but only when you tell it to look. It doesn't spontaneously doubt its own previous output, and in most cases, it will say that everything has been implemented and tested thoroughly. To create the adversarial audit, I have worked with the agent to read and compile the literature about what kind of steering intructions and methods tend to work well with coding agents. As a result, we created a manually invoked instruction document that I invoke whenever something smells fishy. Works pretty well so far.

Delegation as the Core Skill

Here's what I think most people miss. Using these tools well is not about prompt engineering or knowing the right magic words. It's about delegation and evaluation which are the same skills you need to manage people, applied to a machine.

I write instruction sets loaded by task phase: thinking, researching, implementing, reviewing. I run adversarial audits where the agent attacks its own work. I enforce checklists. I push back when the abstraction is wrong. It's less like pair programming and more like being a tech lead for a very fast, very literal engineer who occasionally produces brilliant work and occasionally tries to ship without testing.

The evaluation part is critical and underappreciated. When the agent writes a proof, I need to know enough math to verify it. When it refactors a data mapping, I need to understand the schema well enough to judge whether the new abstraction is better. When it dispatches cloud jobs, I need to understand the API limits to know if the retry logic makes sense. I think I now understand what people meant when they said "AI amplifies your existing expertise". If you don't have the expertise, it amplifies your ability to produce confident-looking garbage.

I wrote proofs, shipped a production API, dispatched thousands of training jobs, and debugged infrastructure WITHIN the same week and while training for a triathlon. A few months ago, that would have been a month of just the engineering work. The tools are crude, context windows degrade, and the agent drifts without anchoring. But for the kind of work where the hard part is the science and the design, and not the typing, it's already a different game for me. And to think I've only started investing into learning agentic coding this January.....

Cheers, see yall next time!

Hi folks, while digging into the state of OVB (Omitted Variable Bias) distribution estimation, I've asked for help from Gemini (Deep Research) and enjoyed what it summarized for me.

Disclaimer: Below is generated by Gemini (errors may exist; but I don't care and I like this deep dive).

Introduction: The Pervasive Challenge of Omitted Variable Bias in Causal Inference.

Omitted variable bias (OVB) stands as a significant impediment to the accurate estimation of causal effects in statistical modeling.1 This bias arises when a statistical model fails to incorporate one or more relevant variables that are correlated with both the independent variable(s) of interest and the dependent variable.3 In essence, the absence of these crucial factors leads the model to misattribute the influence of the missing variable to the variables that are included in the analysis.3 Consequently, the estimated coefficients of the included variables become unreliable, potentially overestimating or underestimating the true causal effects.3 The validity of research findings is thus compromised when important variables are excluded from the model.3

The fundamental issue at the heart of OVB is the entanglement of the omitted variable's impact with the estimated effects of the included variables.3 When a variable that influences the outcome is also correlated with an independent variable in the model, the model mistakenly attributes the changes in the outcome driven by the omitted variable to the included one. For instance, in examining the effect of education on salary, if an individual's ability is not accounted for (perhaps due to measurement challenges), and if ability is correlated with both education level and earning potential, then the estimated effect of education on salary might be inflated by the influence of ability.3 This highlights how the model can mistakenly attribute the effect of the missing variable to the included variables. The pervasive nature of OVB stems from the practical difficulties in identifying and measuring all potential confounding factors in real-world scenarios, particularly within the social and behavioral sciences where numerous interconnected elements can affect outcomes.6

Omitted variable bias arises under specific conditions. Firstly, the excluded variable must have a genuine impact on the dependent variable.3 Secondly, this omitted variable must exhibit a correlation with at least one of the independent variables included in the model.3 The direction of this bias, whether it leads to an overestimation or underestimation of the causal effect, is contingent upon the signs of these correlations.4 A positive correlation between the omitted variable and both the included variable and the outcome typically results in an upward bias. Conversely, differing signs in these correlations can lead to a downward bias.4 The presence of OVB also violates a core assumption of Ordinary Least Squares (OLS) regression, which posits that the error term should be uncorrelated with the regressors.4 When OVB is present, the omitted variable's effect becomes part of the error term, which is then correlated with the included regressors, leading to biased and inconsistent OLS estimators.4

Given that the precise magnitude and even the direction of OVB are often unknown in practical research, estimating the potential distribution of this bias becomes paramount.3 Understanding this distribution allows for a more nuanced appreciation of the uncertainty surrounding causal estimates.3 This deeper understanding is essential for drawing more reliable conclusions from empirical investigations and for making informed decisions based on the estimated causal effects.5 A singular point estimate of bias offers limited insight into the range of plausible true effects. By estimating the distribution, researchers can quantify the inherent uncertainty and assess the robustness of their findings across various potential scenarios of omitted confounding. This distribution enables the construction of confidence intervals that more accurately reflect the uncertainty introduced by the possibility of omitted variable bias.

Analytical Frameworks for Understanding Omitted Variable Bias.

The mathematical formulation of omitted variable bias in linear regression models provides a foundational understanding of this issue. In a scenario where a simple linear regression of an outcome variable (y) on an independent variable (x) is conducted, and a relevant variable (z) is omitted from the model, the estimated coefficient associated with (x) will be biased.4 This bias can be expressed mathematically as a function of the correlation between (x) and (z), and the effect that (z) has on (y).4 Specifically, if the true underlying relationship is represented by the model y = bx + cz + u (where u is the error term), but the estimated model omits z, taking the form y = ax + v (where v is the error term in the misspecified model), then the expected value of the estimated coefficient a is E[a] = b + cf. Here, f represents the coefficient obtained from a regression of the omitted variable z on the included variable x.4

This mathematical expression for the bias, cf, clearly illustrates that omitted variable bias will only be present when two conditions are met.4 First, c, which represents the true effect of the omitted variable z on the dependent variable y, must be non-zero. If z has no impact on y, then its omission will not bias the estimated effect of x. Second, f, which captures the correlation between x and z (as indicated by the coefficient in the regression of z on x), must also be non-zero. If x and z are uncorrelated, then the variation in x does not systematically coincide with the variation in z, and consequently, the effect of z will not be wrongly attributed to x.

The magnitude and direction of the correlations between the omitted variable, the included variables, and the outcome are critical determinants of the extent and nature of the bias.4 A positive correlation between the omitted variable and both the included variable and the outcome will lead to an upward bias, resulting in an overestimation of the included variable's effect.4 Conversely, if the signs of these correlations differ (e.g., a positive correlation between the omitted variable and the included variable, but a negative correlation between the omitted variable and the outcome, or vice versa), the resulting bias will be downward, leading to an underestimation of the included variable's effect.4 Understanding the hypothesized relationships between a potential omitted variable and the other variables in the model allows researchers to predict the likely direction of the bias, even in the absence of precise knowledge about its magnitude.3 This process is often referred to as "signing the bias," where researchers use logical reasoning to determine whether the effect of an included variable is likely to be overestimated or underestimated due to the omission of a specific confounder.3

While analytical formulas provide a way to quantify the bias in linear models, their practical application in estimating the distribution of OVB is often limited.4 These formulas typically require knowledge of the true relationships involving the omitted variable, which, by definition, is unobserved. However, researchers can leverage logical reasoning, domain expertise, and insights from prior research to make informed assumptions about the signs and potential magnitudes of the correlations involving the omitted variable.3 This allows for a qualitative assessment of the potential bias. Although exact analytical estimation of the OVB distribution is often not feasible, these analytical frameworks are crucial for establishing a solid theoretical understanding of the problem. They lay the groundwork for the development and application of more advanced methods, such as sensitivity analysis and bounding approaches, which aim to address the challenges posed by unobserved confounding.

Sensitivity Analysis: Assessing the Distribution of Causal Effects Under Potential OVB.

Sensitivity analysis offers a suite of techniques specifically designed to evaluate the robustness of causal inferences in the face of potential omitted variable bias.1 These methods aim to assess how different assumptions about the characteristics and relationships of unobserved confounders might influence the estimated causal effect. Rather than assuming the absence of such confounders, sensitivity analysis explores a range of plausible scenarios regarding their existence and impact, thereby providing a more nuanced understanding of the uncertainty surrounding causal estimates.1 Often, these techniques involve parameterizing the potential confounder based on its hypothesized associations with both the treatment variable and the outcome variable.15

One key aspect of sensitivity analysis involves quantifying the strength of confounding that would be necessary to alter the conclusions of a study. Measures such as the "robustness value" (RV) serve this purpose.12 The robustness value describes the minimum strength of association, often measured using partial R², that an unobserved confounder would need to exhibit with both the treatment and the outcome to change the substantive findings of the research.12 Similarly, the partial R² of the treatment with the outcome, conditional on any observed covariates, provides insight into how strongly a confounder that explains all the remaining variation in the outcome would need to be associated with the treatment to completely eliminate the estimated effect.12 These measures offer intuitive benchmarks for evaluating the plausibility of omitted variable bias. By quantifying the required "strength" of an unobserved confounder to overturn the results, researchers can leverage their domain-specific knowledge to assess whether such a strong confounder is likely to exist in their particular context. If only a very weak and implausible confounder is sufficient to change the conclusions, the initial findings are considered fragile. Conversely, if a very strong and unlikely confounder would be needed to alter the results, the findings are deemed more robust.

Graphical tools also play a significant role in sensitivity analysis, facilitating the visualization of how point estimates and confidence intervals might be affected by potential omitted variable bias.15 Sensitivity contour plots, for example, can illustrate how the estimated causal effect (whether represented as a point estimate, a t-value, or a confidence interval) varies across different combinations of partial R² values between the potential confounder and the treatment, and between the confounder and the outcome.15 These plots allow researchers to readily observe the regions where the estimated effect remains statistically significant and the regions where it might become insignificant under different confounding scenarios, providing a visual assessment of the robustness of their findings. Additionally, "extreme-scenario" sensitivity plots can be employed to explore the potential impact of hypothetical confounders that explain a substantial portion of the variance in the outcome that is not already accounted for by the included variables.15 These visual aids enhance the understanding and communication of the sensitivity of research results to the potential presence of omitted variable bias, allowing researchers to effectively convey the range of plausible effects under various confounding conditions.

Bounding Approaches: Defining Plausible Ranges for Causal Effects Considering OVB.

Bounding approaches represent a class of methods that aim to define a plausible range or distribution for the true causal effect by explicitly considering the potential influence of unobserved confounders.1 Unlike sensitivity analysis, which explores a variety of scenarios, bounding techniques seek to establish upper and lower limits within which the true causal effect is likely to lie, given certain assumptions about the nature and magnitude of the unobserved confounding.1 These methods often rely on making assumptions about the maximum possible explanatory power that the omitted variables could have on both the treatment and the outcome.1

One common strategy within bounding approaches involves utilizing plausibility judgments regarding the maximum explanatory power of omitted variables to directly bound the potential bias.1 Researchers can leverage their subject-matter expertise and insights from previous research to make informed judgments about the maximum proportion of variance in the outcome and/or the treatment that could realistically be explained by any unobserved confounder.1 These plausibility judgments then serve as the basis for deriving bounds on the magnitude of the omitted variable bias. Consequently, these bounds on the bias can be used to establish a range of plausible values for the true causal effect.1 The credibility of these bounds is intrinsically linked to the validity and justification of the initial plausibility judgments. Researchers must carefully articulate the rationale behind their assumptions, drawing upon existing theoretical frameworks and empirical evidence to support the chosen limits on the explanatory power of potential unobserved confounders. If the assumed maximum explanatory power is set too low, the resulting bounds might be overly narrow and exclude the true effect. Conversely, if the assumed power is too high, the bounds might become so wide that they offer little practical information.

Covariate benchmarking serves as a valuable tool to inform the assumptions made in bounding approaches.8 This technique involves using the observed relationships between the covariates included in the model and the treatment and outcome variables to make inferences about the potential strength of any unobserved confounder.8 By comparing the explanatory power of the observed covariates with what is deemed plausible for an unobserved confounder, researchers can develop more data-driven and justifiable bounds on the potential bias.8 For example, a researcher might argue that an unobserved confounder is unlikely to have a stronger relationship with the treatment or the outcome than the strongest observed covariate in the dataset. This comparison provides an empirical anchor for the plausibility judgments, moving beyond purely subjective assessments. Covariate benchmarking can involve comparing the partial R² of the observed covariates with the treatment and outcome to establish a reasonable upper limit for the potential partial R² of an unobserved confounder. This approach allows for a more informed and defensible range of plausible causal effects, accounting for the potential influence of unobserved variables in a way that is grounded in the observed data.

Simulation-Based Methods: Estimating the Distribution of OVB Through Simulated Scenarios.

Simulation-based methods, particularly Monte Carlo simulations, offer a powerful approach to understanding and estimating the distribution of omitted variable bias.5 These methods involve the creation of artificial datasets where the true underlying causal relationships, including the influence of a specific omitted variable, are precisely known by the researcher.5 By repeatedly generating data under various pre-specified scenarios for the omitted variable and then estimating the causal effect of interest while deliberately omitting this variable from the statistical model, researchers can empirically assess the resulting distribution of bias.21

Simulations provide a controlled environment to investigate the behavior of estimators in the presence of OVB under different conditions. Researchers can systematically vary parameters such as the strength of the relationship between the omitted variable and the included variables, the magnitude of the omitted variable's effect on the outcome, the sample size of the simulated data, and the correlation structure among the variables.21 By observing how these variations affect the distribution of the estimated causal effect (specifically, the bias), researchers can gain valuable insights into when OVB is likely to be most severe and how it might distort the results of their analyses. For instance, simulations can demonstrate that the magnitude of OVB tends to increase with stronger correlations between the omitted and included variables, and with a larger effect of the omitted variable on the outcome.21 Furthermore, simulation studies can be designed to evaluate the effectiveness of different strategies aimed at mitigating OVB, such as the use of proxy variables or instrumental variables, under various simulated conditions.20 This allows for a comparative assessment of the performance of different methodological approaches in the presence of unobserved confounding.

Beyond understanding the conditions that exacerbate OVB, simulation-based methods play a crucial role in validating other analytical and sensitivity analysis techniques.7 The empirical distribution of bias obtained from a well-designed simulation study can be compared with the theoretical predictions derived from analytical formulas for OVB. Similarly, simulation results can be used to assess the coverage properties of confidence intervals generated by sensitivity analysis methods. If a simulation study consistently shows that the observed bias falls within the bounds predicted by a particular sensitivity analysis technique, it provides empirical support for the reliability and validity of that technique in real-world applications where the true bias is unknown. This process of validation through simulation helps to build confidence in the accuracy and robustness of the different methods used to address the challenges posed by omitted variable bias in causal inference.

Bayesian Methods: Incorporating Uncertainty from Unobserved Confounding into Causal Inference.

Bayesian methods offer a natural framework for incorporating the uncertainty arising from unobserved confounding directly into the estimation of causal effects.25 A key feature of the Bayesian approach is the explicit modeling of uncertainty through the use of prior distributions.25 In the context of omitted variable bias, prior distributions can be specified for the parameters of the unobserved confounder itself, or directly on the potential bias introduced by its omission.26 This leads to a posterior distribution for the causal effect of interest that inherently reflects the uncertainty stemming from the presence of these unmeasured factors.26

The Bayesian perspective treats the true causal effect as a random variable with an associated probability distribution, which readily accommodates the uncertainty inherent in the presence of unobserved confounders.26 Instead of providing a single point estimate of the causal effect, Bayesian analysis yields a full posterior distribution. This distribution represents the probability of different values of the causal effect being the true value, given the observed data and the prior beliefs about the unobserved confounding. Consequently, researchers can make probabilistic statements about the range of plausible causal effects, such as the probability that the effect falls within a specific interval. The specification of prior distributions is a crucial step in Bayesian sensitivity analysis for OVB.26 Researchers can choose priors that reflect their existing knowledge or beliefs about the likely magnitude and direction of the bias caused by unobserved confounding, drawing upon domain expertise, previous research findings, or theoretical considerations.26 These priors can be informative, reflecting strong prior beliefs, or non-informative, indicating a lack of strong prior knowledge about the potential bias.26

Bayesian sensitivity analysis involves examining how the resulting posterior distribution of the causal effect changes under different plausible prior distributions for the unobserved confounder or the bias it introduces.16 This process allows researchers to assess the extent to which their conclusions about the causal effect are driven by their prior assumptions about the unobserved confounding, rather than solely by the information contained in the observed data.16 If the posterior distribution of the causal effect is found to be highly sensitive to the choice of prior, it suggests that the data alone do not provide strong enough evidence to overcome the initial uncertainty introduced by the potential for unobserved confounding. In such cases, researchers might need to consider alternative study designs or seek additional data to strengthen their causal inferences. Conversely, if the posterior distribution remains relatively stable across a range of plausible prior specifications, it indicates that the findings are more robust to the uncertainty surrounding unobserved confounders.

Machine Learning Techniques: Modeling and Estimating the Distribution of OVB in Complex Settings.

Machine learning techniques have emerged as powerful tools for addressing the challenges of omitted variable bias, particularly in complex settings involving high-dimensional data and non-linear relationships.1 These methods offer flexible and often non-parametric approaches to modeling the intricate relationships between variables, which can be especially beneficial when dealing with a multitude of potential confounders.1 Machine learning algorithms can be effectively used to estimate the conditional expectation function of the outcome variable, a crucial component in many modern causal inference methodologies.1 The ability of machine learning to handle non-linearities and high-dimensional datasets makes it a valuable asset in addressing OVB in scenarios where traditional parametric methods might fall short. For instance, in situations where the effects of confounders are not simply additive or linear, machine learning models can capture these more complex interactions.

While machine learning excels at predictive tasks, quantifying the uncertainty associated with causal effect estimates, especially in the presence of omitted variable bias, remains an active area of research and development.27 Some machine learning-based causal inference methods, such as those employing Bayesian Additive Regression Trees (BART), are capable of providing estimates of the uncertainty inherent in the estimated causal effects.27 Additionally, techniques like Double Machine Learning (DoubleML) are incorporating methods for conducting sensitivity analysis specifically with respect to omitted variable bias within the framework of machine learning models.17 Despite the predictive power of many machine learning algorithms, their inherent "black box" nature can sometimes make it challenging to understand precisely how they are accounting for confounding variables and to rigorously assess the uncertainty in their causal estimates. Consequently, ongoing efforts are focused on integrating methods like bootstrapping and Bayesian approaches with machine learning to enhance the interpretability and uncertainty quantification of causal inferences derived from these powerful techniques.

Recent advancements in the field have focused on extending the theoretical framework of omitted variable bias to encompass non-linear models, often leveraging the estimation and inference capabilities of machine learning.1 These developments provide researchers with tools to bound the potential magnitude of bias in complex, non-linear models by employing plausibility judgments regarding the explanatory power of omitted variables.1 This is particularly significant for real-world applications where the relationships between variables are frequently non-linear and multifaceted. By providing methods to assess and bound OVB in the context of flexible machine learning models, researchers can move beyond the limitations of traditional linear models and strive for more reliable and accurate causal effect estimates across a wider spectrum of research questions and data structures.

Conclusion: Navigating the Challenges of Estimating OVB Distribution and Future Research Directions.

Estimating the distribution of omitted variable bias presents a multifaceted challenge in causal inference, and various approaches have been developed to address this issue, each with its own strengths and limitations. Analytical methods offer a fundamental understanding of the mechanisms underlying OVB, but their practical utility in estimating the distribution is often limited by the unobservable nature of the confounders. Sensitivity analysis provides a valuable framework for exploring how different assumptions about unobserved confounders might affect causal estimates, but it does not yield a single "corrected" estimate or distribution of the bias. Bounding approaches aim to define a plausible range for the true causal effect by making assumptions about the maximum influence of omitted variables, but the validity of these bounds hinges on the credibility of these assumptions. Simulation-based methods allow for controlled experimentation and the empirical assessment of bias distributions under specific scenarios, but the results are contingent on the chosen data-generating process. Bayesian methods offer a natural way to incorporate uncertainty through prior distributions, but the resulting inferences can be sensitive to the specification of these priors. Finally, machine learning techniques provide powerful tools for modeling complex relationships, but quantifying uncertainty in causal effect estimates, especially in the presence of OVB, remains an evolving area.

Despite the advancements in these methodologies, the fundamental challenge of quantifying the impact of unobserved confounders persists. The very nature of an omitted variable – being unobserved – makes direct estimation of its properties and its relationships with other variables inherently difficult. Making credible plausibility judgments in bounding approaches and selecting appropriate prior distributions in Bayesian methods often involves a degree of subjectivity. Furthermore, extending the analysis of OVB to more complex non-linear models and ensuring the interpretability of causal inferences derived from machine learning remain active areas of research.

Future research should focus on developing more robust and practical methods for estimating the distribution of omitted variable bias. This includes exploring data-driven approaches to inform the plausibility judgments in bounding techniques and the specification of prior distributions in Bayesian frameworks, potentially leveraging auxiliary data sources or meta-analytic evidence. Further investigation into extending OVB analysis to non-linear and high-dimensional settings, particularly within the context of machine learning, is also crucial. The development of more user-friendly software tools and clear guidelines for implementing and interpreting the various methods for estimating OVB distribution would greatly benefit applied researchers. Additionally, exploring novel data sources and study designs that are inherently less susceptible to omitted variable bias remains an important avenue. Finally, research into methods for combining information from different sensitivity analysis and bounding approaches could lead to more robust and comprehensive assessments of the uncertainty introduced by omitted variable bias in causal inference.

Works cited

1. Long Story Short: Omitted Variable Bias in Causal Machine Learning - arXiv, accessed April 3, 2025, https://arxiv.org/html/2112.13398v5
2. LONG STORY SHORT: OMITTED VARIABLE BIAS IN CAUSAL MACHINE LEARNING - Carlos Cinelli, accessed April 3, 2025, https://carloscinelli.com/files/Chernozhukov%20et%20al%20(2021)%20-%20OVB%20for%20ML.pdf
3. What Is Omitted Variable Bias? | Definition & Examples - Scribbr, accessed April 3, 2025, https://www.scribbr.com/research-bias/omitted-variable-bias/
4. Omitted-variable bias - Wikipedia, accessed April 3, 2025, https://en.wikipedia.org/wiki/Omitted-variable_bias
5. Think before you act: Facing omitted variable bias - arXiv, accessed April 3, 2025, https://arxiv.org/html/2501.17026v1
6. Causal inference with observational data and unobserved confounding variables - bioRxiv, accessed April 3, 2025, https://www.biorxiv.org/content/10.1101/2024.02.26.582072v1.full-text
7. Examining Management Research With the Impact Threshold of a Confounding Variable (ITCV) - Terry College of Business, accessed April 3, 2025, https://www.terry.uga.edu/wp-content/uploads/Busenbark-Yoon-Gamache-and-Withers-2022.pdf
8. www.econstor.eu, accessed April 3, 2025, https://www.econstor.eu/bitstream/10419/283961/1/1353.pdf
9. 6.1 Omitted Variable Bias | Introduction to Econometrics with R, accessed April 3, 2025, https://www.econometrics-with-r.org/6.1-omitted-variable-bias.html
10. Omitted Variable Bias: Definition, Avoiding & Example - Statistics By Jim, accessed April 3, 2025, https://statisticsbyjim.com/regression/omitted-variable-bias/
11. Omitted Variable Bias Framework for Sensitivity Analysis of Instrumental Variables | Biometrika | Oxford Academic, accessed April 3, 2025, https://academic.oup.com/biomet/advance-article/doi/10.1093/biomet/asaf004/7978915
12. Making Sense of Sensitivity: Extending Omitted Variable Bias ..., accessed April 3, 2025, https://academic.oup.com/jrsssb/article-abstract/82/1/39/7056023
13. Bias and Sensitivity Analysis When Estimating Treatment Effects from the Cox Model with Omitted Covariates - PMC, accessed April 3, 2025, https://pmc.ncbi.nlm.nih.gov/articles/PMC4230475/
14. An Omitted Variable Bias Framework for Sensitivity Analysis of Instrumental Variables, accessed April 3, 2025, https://sbe.org.br/anais/45EBE/econometria/45_EBE_paper_2.pdf
15. Making Sense of Sensitivity: Extending Omitted Variable Bias - Carlos Cinelli, accessed April 3, 2025, https://carloscinelli.com/files/Cinelli%20and%20Hazlett%20(2020)%20-%20Making%20Sense%20of%20Sensitivity.pdf
16. Intermediate Causal Inference - Andy Eggers, accessed April 3, 2025, https://andy.egge.rs/teaching/oss/ici_2019/eggers_ICI_slides_v1.pdf
17. 10. Sensitivity analysis — DoubleML documentation, accessed April 3, 2025, https://docs.doubleml.org/stable/guide/sensitivity.html
18. Carlos Cinelli - Making Sense of Sensitivity: Extending Omitted Variable Bias - YouTube, accessed April 3, 2025, https://www.youtube.com/watch?v=HZvZYywf3fg
19. LONG STORY SHORT: OMITTED VARIABLE BIAS IN CAUSAL MACHINE LEARNING - Carlos Cinelli, accessed April 3, 2025, https://carloscinelli.com/files/Chernozhukov%20et%20al%20-%20OVB%20for%20ML.pdf
20. (PDF) Investigating Omitted Variable Bias In Regression Parameter Estimation: A Genetic Algorithm Approach - ResearchGate, accessed April 3, 2025, https://www.researchgate.net/publication/39728492_Investigating_Omitted_Variable_Bias_In_Regression_Parameter_Estimation_A_Genetic_Algorithm_Approach
21. r - Omitted Variable Bias in Linear Regression - Simulation - Cross ..., accessed April 3, 2025, https://stats.stackexchange.com/questions/142693/omitted-variable-bias-in-linear-regression-simulation
22. Monte Carlo simulation on Omitted Variable Bias : r/RStudio - Reddit, accessed April 3, 2025, https://www.reddit.com/r/RStudio/comments/ebg6jg/monte_carlo_simulation_on_omitted_variable_bias/
23. A COMPARISON OF SOME ESTIMATION METHODS FOR HANDLING OMITTED VARIABLES : A SIMULATION STUDY - Diva Portal, accessed April 3, 2025, https://uu.diva-portal.org/smash/get/diva2:1439186/FULLTEXT01.pdf
24. Monte-Carlo simulation for omitted variable bias, accessed April 3, 2025, https://capone.mtsu.edu/eaeff/462/wpp/MonteCarloOmitVarb.html
25. Bayesian causal inference: A critical review and tutorial (Standard Format) - YouTube, accessed April 3, 2025, https://www.youtube.com/watch?v=7Cwl6DgL64o
26. A Practical Introduction to Bayesian Estimation of Causal Effects ..., accessed April 3, 2025, https://pmc.ncbi.nlm.nih.gov/articles/PMC8640942/
27. Bayesian Causal Inference for Observational Studies with Missingness in Covariates and Outcomes | Biometrics | Oxford Academic, accessed April 3, 2025, https://academic.oup.com/biometrics/article/79/4/3624/7587588

Hey there, folks! Billy here, back with another tale from the trenches of applied data science. Today, we’re tackling the beast known as the mass point—a pesky cluster of values that throws off our beloved linear regression, especially after a log transformation. Spoiler alert: it’s like having a party where everyone shows up in the same outfit, and the model just can’t decide if it’s a fashion statement or a malfunction!

What the Heck Is a Mass Point?

Imagine you have a dataset where a huge number of observations are identical—say, a whole crowd buying a $1 product. When you log-transform your sales (because we love making data “normal”), those $1’s all morph into a big, fat zero (since log(1) = 0). Now your data is suddenly dominated by a mass of zeros.

For the ones who prefer visuals: The "Mass Point" Party

• Left: A histogram of the original data showing a huge spike at T=1.
• Right: The same data after log transformation, now with a towering spike at 0.
  (See Figure 1 below.)

Why Linear Models Struggle with Mass Points

Linear regression relies on several key assumptions to produce reliable results:​

1. Linearity: The relationship between the independent variable(s) and the dependent variable is linear.​
2. Independence of Errors: The residuals (errors) are independent of each other.​
3. Homoscedasticity: The residuals have constant variance across all levels of the independent variable(s).​
4. Normality: The residuals are approximately normally distributed.​

When a dataset contains a mass point—a large concentration of identical values—it can violate these assumptions, particularly affecting the normality and homoscedasticity of residuals (i.e. the prediction errors of the regression model have a constant variance/spread across all levels of the independent variables, rather than changing or forming patterns).​

Example: Consider a dataset where a significant number of observations have the value T=1. Applying a logarithmic transformation (commonly used to normalize data) results in log⁡(1)=0, creating a spike at zero in the transformed data. This concentration can lead to:​

1. Non-Normal Residuals: The residuals may no longer follow a normal distribution due to the disproportionate influence of the mass point. For example, when a mass point exists, it can skew the residuals by introducing asymmetry or changing their spread, violating normality assumptions. Essentially, the mass point pulls the residuals toward itself, distorting their distribution instead of keeping them balanced and normally spread around zero.
2. Heteroscedasticity: The variance of residuals may not remain constant, as the model struggles to account for the density at the mass point compared to other values.​ Ensuring homoscedasticity is important because if the variance of residuals changes—especially due to a mass point—it means the model struggles to maintain consistent accuracy across different data ranges. This can result in biased standard errors (making hypothesis tests (e.g., t-tests) unreliable), inefficient estimates (meaning the model’s predictions might not be as precise as they could be) and misleading confidence intervals (which can lead to incorrect conclusions about the significance of variables).

These issues can cause the linear model to produce biased or inefficient estimates, as it attempts to fit a linear relationship to data that doesn't adhere to its foundational assumptions.

For the ones who prefer visuals: Regression vs. Mass Points
A scatter plot of our log-transformed data with a linear regression line that awkwardly skirts around the massive cluster at 0 (our fitted model gets skewed/biased towards the point mass):

Models That Know How to Party

When linear regression meets a mass point, it’s like inviting everyone to a black-tie event and having half the guests show up in the same neon T-shirt—it just doesn’t blend well. Instead, we can adopt alternative models that know how to handle these quirks and let our data have its own kind of party. Here are three possible solutions to fix the mass point problem (with their pros and cons):

Wrapping It All Up

When linear regression is forced to handle mass points, it’s like watching a classical dancer attempt breakdancing—awkward and off-beat. By adopting alternative approaches (hurdle, mixture, or zero-inflated models), we let the data express its natural quirks rather than forcing it into a mold it just doesn’t fit.

Next time you see a giant spike at zero in your log-transformed data, remember: sometimes you need a bouncer, a cocktail mix, or even a vending machine model to keep the party (and the predictions) lively.

Cheers, Billy!

Appendix

Impact of Heteroscedasticity on Linear Regression:

1. Biased Standard Errors: When heteroscedasticity is present, the standard errors of the coefficient estimates become unreliable. This unreliability can lead to inaccurate confidence intervals and hypothesis tests, increasing the likelihood of Type I or Type II errors. ​
2. Inefficient Estimates: Although ordinary least squares (OLS) estimators remain unbiased in the presence of heteroscedasticity, they are no longer the best linear unbiased estimators (BLUE). This inefficiency means that the estimates have higher variance than necessary, making them less precise. ​

Importance of Efficiency in Linear Regression

Efficiency in the context of linear regression refers to obtaining estimators with the smallest possible variance. Efficient estimators are desirable because:​

1. Precision: More efficient estimators provide tighter confidence intervals, enhancing the reliability of the estimates.​
2. Statistical Power: With increased precision, statistical tests are more likely to detect true effects when they exist, reducing the risk of Type II errors.​

This one summarizes what I've captured from Chapter 14 of "Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction" book by Guido W. Imbens and Donald B. Rubin.

Few shortcuts I've used:

• T - shortcut for treatment group
• C - shortcut for control group

Covered sections and their summary:

• 14.1 Introduces why adjusting for differences in the covariate distributions is needed
• 14.2 Goes over a case with two univariate distributions
• 14.3 Goes over a case with two multivariate distributions
• 14.4 Explains the role of propensity of scores in assessing the overlap of covariate distributions
• 14.5 Provides a measure to assess if each treatment/control unit has an identical non-treated/treated twin
• 14.6 Provides examples (skipped)

1. Why do we need to adjust for differences in the covariate distributions?

If there is a region in the covariate space where T has relatively few units/samples or C has relatively few units/samples, our inferences in that region will largely depend on extrapolation, thus will be less credible compared to inferences of regions where both T and C have substantial overlap in the covariates distribution.

Example:
Covariate space #1:

• T has 5 males with age<=18
• C has 55 males with age<=18.

Covariate space #2:

• T has 55 males age>18 and age<=30
• C has 60 males age>18 and age<=30

In the above example, our inferences for the covariate space #2 will be more credible than our inferences for the covariate space #1.

To note, even in cases when there is there is no confounding (unconfoundedness), this is still a fundamental issue. However, if we have a completely randomized experiment, we can expect the distribution of covariates to be similar per definition, and thus, less risks of stumbling upon this issue.

2. Assessing overlap of univariate distributions

1. Comparing differences in location of two distributions

Given two univariate probability distributions fc​(x) and ft​(x) with means μc​ and μt​, and variances σc2​ and σt2​, we can estimate the differences in locations of the two distributions with:

, which is a normalized measure of difference of two distributions. To note, this is different from the t-statistic that tests whether the data contain sufficient info to support the hypothesis that two covariate means in T and C are different:

To our purposes, we don’t care about testing if we have enough data to check if means are different between T&C. Instead, we care about understanding if the differences between the distributions are so large that we need to fix the issue, and at the same time, check what kind of adjustment methods are required for the problem at hand.

2. Comparing measures of dispersion of two distributions

The log difference in the standard deviations helps to understand the difference in dispersion:

, we take the log because its more normally distributed compared to simple difference or ratio of standard deviations.

3. Comparing the tail overlaps of two distributions

To understand the overlap of covariate distributions in the tail regions, we can calculate the fraction of treatment units that exist in the tails of the distribution of covariate values in the control.

One method is to choose an arbitrary tail boundary, e.g. α=0.05 and calculate the probability mass of T’s covariate distribution outside the α/2 and 1−α/2 quantiles of the C’s covariate distribution:

, where Ft​ and Fc​ represent the CDFs of the T and C covariates’ distributions. We can do the same for C, where we calculate the probability mass of C’s covariate distribution outside the α/2 and 1−α/2 quantiles of the T’s covariate distribution:

The intuition here is that it is relatively easy to impute outcomes values of either T or C units in the dense regions of the distribution (e.g. near the mean of a normal distribution where majority of samples are concentrated), while it is harder at the tail regions where only few units exist. One can refer to the following two distributions as an example of two differently dispersed covariates’ distributions:

Last but not least, here’s how to interpret the values:

• πcα​=πtα​=α happens for a completely randomized experiment, where only α∗100% of units have covariate values that make us cry (i.e. difficult to impute the missing potential outcomes of a group).
• if πtα​>α, it will be relatively difficult to impute/predict the missing potential outcomes for the control units (more treatment units exist in the tail region of the control group’s covariate distribution).
• if πtα​>α, it will be relatively difficult to impute/predict the missing potential outcomes for the control units (more treatment units exist in the tail region of the control group’s covariate distribution).

3. Assessing overlap of multivariate distributions

Given K covariates, we could compare the distributions of T & C iteratively one-by-one using the above approach. A good way is to start with the covariates for which you have a priori belief that it is highly associated with the outcome (i.e. ensuring the importatnt covariates are okay).

Using Mahalanobis distance to compare differences of multivariate distributions

In addition to above metrics that can be used for univariate distributions, there is a neat multivariate summary measure that captures the difference in locations between two distributions similar to the one mentioned in the first part of section 2. It leverages the Mahalanobis distance and takes as input the 2 by K-dimensional vector of distribution means of each covariate μc​ and μt​:

, where

are the covariance matrices of distribution means for C & T, respectively. Intuitively, a larger Mahalanobis distance tells us that the distributions are further apart in multivariate space, considering both location and spread.

Using propensity scores to compare distributions

We can use propensity scores to assess the balance of covariate distributions is that any difference in covariate distributions also shows up in the difference of propensity score distributions. In principle,  difference in covariate distributions leads to difference in the expected propensity scores of T&C, i.e. the average propensity scores. So if there’s a non-zero difference in the propensity score distributions for T&C, it also implies there is a difference in the covariate distributions for T&C.

Since it is much less work to analyze the differences of univariate distributions compared to multivariate ones, leveraging propensity scores and their distributions which is a univariate distribution to analyze the differences makes things a bit simpler.

How does it work? Consider t(x) to be the true propensity score (assuming we know the treatment assignment mechanism) and l(x) to be the linearized propensity score (log odds ratio) of being in T vs C given covariates x:

Then, we can simply look at the normalized difference in means for the propensity scores of each treatment, where:

are the average values for propensity scores of C & T units, and:

are the variances of the propensity score, which leads us to the estimated difference in average propensity scores scaled by the square root of the average square within-treatment-group standard deviations:

To note, if we are using linearized propensity scores, the propensity score function is scale-invariant, so there is not actually a need to normalize the above by the standard deviations.

Moreover, compared to how we assessed the univariate covariate distributions, there are some points to be aware of:

1. Differences in the covariate distributions are implied by the variation in the propensity scores.
2. If the treatment assignment mechanism is biased some way then it is possible that the covariate distributions are similar, but we observe a difference in the propensity score distributions.
3. On another hand, the treatment assignment mechanism might not be biased, and the covariate distributions do indeed differ (implied by the difference in propensity score distributions).
4. If the covariate distributions of two treatments differ, then it must be that the expected value of the propensity score in T is larger than the expected value of the propensity score in the C (or vice-versa).

As a result, we can understand that differences in covariate distributions in T vs C imply, and are implied by, the differences in the average value of the propensity scores of T & C. Proof is captured in the book Chapter 14.4.

4. Estimating if T/C units have comparable twins in C/T

Consider a unit i with treatment Wi​. For this unit i we can determine if there is a comparable twin with treatment Wi^​=1−Wi​ such that the difference in propensity scores l(Xi​)−l(Xiˉ​) is less than or equal to lu, where lu is an upper threshold implying the difference in propensity scores is less than 10%.

Intuitively, if there is a similar unit with opposite treatment and a similar propensity score, we may be able to obtain trustworthy estimates of causal effects without any extrapolation or additional work. However, if there are many such units without twins, it will be difficult to obtain credible estimates of causal effects, no matter what methods we use. A common method to solve this, given we have enough samples, is to trim the units without twins.

We can estimate the degree of units with close comparison units by defining an indicator variable:

, then for each T & C we can define two overlap measures implying the proportion of units with close comparisons:

Covariance and the dot product are related concepts in mathematics and statistics, particularly in the context of vectors and random variables. Here's how they are connected:

Dot Product

The dot product (also known as the scalar product) of two vectors a and b in n-dimensional space is given by:

This operation results in a single scalar value and provides a measure of the similarity between the two vectors. If the vectors point in the same direction, the dot product is positive and large; if they point in opposite directions, the dot product is negative; if they are orthogonal, the dot product is zero.

Covariance

Covariance is a measure of how much two random variables X and Y change together. It is given by:

Where E denotes the expected value. If the covariance is positive, the variables tend to increase together; if it is negative, one tends to increase when the other decreases.

Relationship

The relationship between covariance and the dot product can be seen in the context of centered random variables. If you consider the centered random variables:

the covariance between X and Y can be interpreted as the dot product of these centered variables in the space of random variables, normalized by the number of observations n:

Here, tilde_X_i and tilde_Y_i are the deviations from the mean of the random variables X and Yrespectively. This sum is analogous to the dot product, where the sum of the products of corresponding elements gives a measure of the overall relationship between the two sets of values.

Summary

In summary, covariance can be viewed as a normalized dot product of centered random variables. Both operations measure how two sets of values relate to each other, with the dot product being a geometric measure in vector space and covariance being a statistical measure in the space of random variables.

Machine Learning (ML) Proof Of Concept (POC) is one of the key phases in ML projects. Iteratively coming up with a better modeling approach, improving data quality, and finally, delivering a minimum viable solution for a grand problem is fascinating and you learn a lot.

At the end of the POC phase, if the results are satisfactory and stakeholders agree to move forward, the ML engineer gets tasked with finding and implementing an appropriate method for deploying the model into production. Depending on the business use case and the scale of the project (and many other factors), he/she will usually select to proceed with one of the following (non-exhaustive):

1. Type 1: Save the trained model artifacts in the backend. Implement an inference logic that uses the trained model. The frontend engineer then requests the predictions from the backend via HTTP.
2. Type 2: Choose to use one of the cloud services (Amazon AWS, Google GCP, Microsoft Azure) for training the model. Transfer the training data to the cloud, train the model and deploy it in the cloud. Set up a cloud endpoint and return predictions via HTTP requests.
3. Type 3 (worst): Implement the training and the inference logic in the backend. At the same time, implement the data extraction, transformation and the loading (ETL) processes in the backend so that the system is able to retrain the model when needed. Try his/her best to setup monitoring, so that one day when things go wrong, he/she can find and fix the problem.
4. Type 4: Use the cloud services to setup everything (ETL, data preprocessing, model training, monitoring, configuration, etc.). The frontend engineer then requests predictions via HTTP endpoint provided by the cloud service.
5. Type 5: Use open-source packages, connect and build everything manually.

There can be many other options too. However the key message here is that ML deployment can take on many forms.

With ever increasing data and the expertise of the people dealing with the data, various ML models are being trained, tested, and deployed into production with unprecedented rates. However, with the increasing number of model deployments, ML practitioners have been experiencing a set of problems related to the unique property of ML-embedded systems - the usage of data. And this is exactly where MLOps comes to the rescue. Similar to the concept/culture of DevOps in the traditional software engineering discipline, the term MLOps refers to the engineering culture and best practices related to deploying and maintaining real-world ML systems.

There are many resources related to MLOps. Some of my favorites are:

1. "Hidden Technical Debt in Machine Learning Systems" - Google (Analyzed in this blog post.)
2. "Machine Learning Guides" - Google
3. "MLOps for non-DevOps folks, a.k.a. “I have a model, now what?!" - Hannes Hapke
4. "From Model-centric to Data-centric AI" - Andrew Ng
5. "Bridging AI's Proof-of-Concept to Production Gap" - Andrew Ng
6. "MLOps: Continuous delivery and automation pipelines in machine learning" - Google
7. "What is ML Ops? Best Practices for DevOps for ML" - Kaz Sato
8. ...and so on.

Perhaps the most popular resource on the above list is the paper called "Hidden Technical Debt in Machine Learning Systems" by Google.

When I first read the paper, I found it to be highly useful in practice and understood that I need to go over this paper whenever I am building ML solutions. So to make my life a bit easier, I have decided to summarize the paper and do it so using simple terms and explanations.

Key points:

1. Developing and deploying ML systems might seem fast and cheap, but maintaining it for the long term is difficult and expensive.
2. Goal of dealing with technical debt is not to add new functionality, but to enable future improvements, reduce errors, and improve maintainability.
3. Data influences ML system behavior. Technical debt in ML systems may be difficult to detect because it exists at the system level rather than the code level.

Complex Models Erode Boundaries

1.  Entanglement

1. CACE principle:
   CACE stands for Changing Anything Changes Everything. It refers to the entangled property of ML-systems, where changing the input distributions of a single feature can lead to changes in the rest of the features. CACE applies to input signals, hyper-parameters, learning settings, sampling methods, convergence thresholds, data selection, and essentially every other possible tweak.
2. Possible solution #1:
   If the problem can be decomposed into sub-problems (disjointed, uncorrelated), train models for each sub-problem and serve ensemble models. In many cases ensembles work well because the errors in the component models are uncorrelated. However, ensemble models lead to strong entanglement: improving an individual model may actually make the system accuracy worse if the remaining errors are more strongly correlated with the other components.
3. Possible solution #2:
   Monitor and detect changes in the prediction behavior in real-time. Use visualization tools to analyze the changes.

2. Correction Cascades (chain reaction)

1. When we have a ready-to-use model m_a for problem A and need to train a new model for a problem A^', it makes sense to train a correction model m^'_a that takes as input the predictions of the model m_a.
2. This dependency can be cascaded further, for example training a model m^''_aa based on the model m^'_a.
3. This dependency structury creates an improvement deadlock. When we try to improve the accuracy of a single model, other models are affected by the change causing system-level issues.
4. Possible solution #1:
   Train and tune the model m_a directly for the problem A^' by adding appropriate features specific to the problem A^'.
5. Possible solution #2:
   Create a new model for the problem A^'.

3. Undeclared Consumers (Please let us know if you are using!)

1. Often, predictions of a model are used in other services.
2. If we don't identify all end users of a model, later in the process, if we make changes to the model, the secret consumers will be affected silently and will face strange issues. Without clear definition of consumers, it becomes difficult and costly to make changes to the model at all.
3. In practice, engineers will choose to use the most accessible signal at hand (e.g. model predictions), especially when deadlines are approaching.
4. Possible solution:
   Setup access restrictions and determine all consumers. Similarly, setting up service-level agreements (SLAs) would also work.

Data Dependencies Cost More than Code Dependencies

1. Unstable Data Dependencies (Can I rely on you?)

1. ML-systems often consume data (input signals) produced by other systems.
2. Some data might be unstable (qualitatively or quantitatively changing over time).
   For example:
   a) Input data to system B is produced by a machine learning model from system A, and system A decides to update its model.
   b) Input data comes from a data-dependent lookup table (e.g. calculates TF/IDF scores or semantic mappings).
   c) Engineering ownership of the input signal is separate from the engineering ownership of the model that consumes it. Engineers who own the input signal can make changes to the data at any time. This makes it costly for the engineers who consume the data to analyze how the change affects their system.
   d) Corrections in the input data can lead to detrimental consequences too! Similar to the CACE principle.
   For example:
   A model that is previously trained on mis-calibrated input signals can start to behave strangely when the input data is recalibrated. This means that although the update was meant to fix a problem, it actually introduced complications to the system.
3. Possible solution:
   Create versioned copy of the input data. Using stable version of the data until the new data is checked and tested will ensure that the system is stable. ***Keep in mind that saving versioned copies of the data means we have to deal with potential data staleness and the cost of maintenance.

2. Underutilized Data Dependencies (How will this feature affect me in the future?)

1. Underutilized data dependency - data that has very small incremental benefit for the model (0.0001% improvement).
2. Example:
   A system introduces new product numbering logic to the system. Since instantly removing the old product numbering logic will lead to disasters (since other components depend on it), engineers decide to keep the new and the old numbering logics for the time being. This way, new products use the new product numbering logic only, but old products have both the new and the old product numbers. Accordingly, the model is retrained using both features continuing to rely on old product numbers for some products. A year later, when the old numbering logic code is deleted, it will be difficult for the maintainers to find what went wrong.
3. Types of underutilized data dependency:
   a) Legacy Features.
   A feature F is included in a model early in its development. Over time, F is made redundant by new features but this goes undetected.
   b) Bundled Features.
   During an experiment, a group of features is evaluated and found to be useful. Because of deadline pressures, the group of features are added to the model together, possibly including features with low value.
   c) ǫ-Features. As machine learning researchers, it is tempting to improve the model accuracy even when the accuracy gain is small and the complexity overhead is high.
   d) Correlated Features. Often two features are strongly correlated, but one is more directly causal. Many ML methods have difficulty detecting this and credit the two features equally, or may even pick the non-causal one. When the correlations disappear in the future, the model will perform poorly.
4. Possible solution:
   Leave-one-feature-out evaluations can be used to detect underutilized dependencies. Regularly performing this evaluation is recommended.

3. Static Analysis of Data Dependencies (Data Catalogs!)

1. In traditional code, compilers and build systems perform static analysis of dependency graphs. Tools for static analysis of data dependencies are far less common, but are essential for error checking, tracking down consumers, and enforcing migration and updates.
2. Annotating data sources and features with metadata (e.g. deprecation, platform-specific availability, and domain-specific applicability), and setting up automatic checks to make sure all annotations are updated helps to resolve the dependency tree (users and systems who use the data).
3. This kind of tooling can make migration and deletion much safer in practice.
4. Google's solution can be found here (Section 8: "AUTOMATED FEATURE MANAGEMENT").

Feedback Loops

When it comes to live ML systems that update their behavior over time , it is difficult to predict how they will behave when they are released into production. This is called analysis debt - the difficulty of measuring the behavior of a model before deployment.

1. Direct Feedback Loops (Learning wrong things)

1. A model may directly influence the selection of its own future training data.
   Example:
   In an e-commerce website a model might recommend 10 different products to a new customer. The customer might choose a product category that he/she needs to purchase at the moment, but which is not related to his/her actual interests. The model captures this and retrains itself, learning the wrong interest and becoming confident that the chosen product category conveys the customer's interest.
2. Possible Solution #1:
   Acquiring customer feedback on the recommendations provided by the system.
3. Possible Solution #2:
   Increasing "exploration" of the model so that it doesn't "exploit" small number of signals (Bandit Algorithms).
4. Possible Solution #3:
   Using some amount of randomization.

2. Hidden Feedback Loops (We were connected???)

1. Hidden Feedback Loops - two systems influence each other indirectly (in a hidden, difficult-to-detect manner).
2. Difficult to detect!
3. Improving one system may lead to changes in the behavior of the other.
4. Hidden feedback loops can also occur in completely disjoint systems.
   Example:
   Scenario of two investment firms when one firm makes changes to its bidding algorithm (improvements, bugs, etc.) and the other firm's model catches this signal and changes its behavior too (might lead to disasters).

ML-System Anti-Patterns

In the real world, ML-related code takes a very small fraction of the entire system. Therefore, it is important to consider other parts of the system and make sure that there are as few system-design anti-patterns as possible.

1. Glue Code (Packages should be replaceable --> APIs)

1. Glue code system design - system with large portion of code written to support the specific requirements of general-purpose packages.
2. Glue code makes it difficult to test alternatives methods or make improvements in the future.
3. If ML code takes 5% and glue code takes 95% of the total code, it makes sense to implement a native solution rather than using general-purpose packages.
4. Possible Solution:
   Wrap general-purpose packages into common APIs. This way we can reuse the APIs and not worry about changing packages (e.g. scikit-learn's Estimator API with common fit(), predict() methods for all models).

2. Pipeline Jungles (Keep things organized)

1. Usually occurs during data preparation.
2. Adding a new data source to the data pipeline 1-by-1, gradually makes it difficult to make improvements and detect errors.
3. Possible Solution #1:
   Think about data collection and feature extraction as a whole. Redesign the jungle pipeline from scratch. It might require a lot of effort, but is an investment worth the price.
4. Possible Solution #2:
   Make sure researchers and engineers are not overly separated. Packages written by the researchers may seem like black-box algorithms to the engineers who use them. If possible, integrate researchers and engineers into the same team (or create hybrid engineers).

3. Dead Experimental Codepaths (Clean your room!)

1. Adding experimental code to the production system can cause issues in the long term.
2. It becomes difficult/impossible to test all possible interactions between codepaths. The system becomes overly complex with many experimental branches.
3. "A famous example of the dangers here was Knight Capital’s system losing $465 million in 45 minutes, because of unexpected behavior from obsolete experimental codepaths." (source)
4. Possible Solution:
   "Periodically reexamine each experimental branch to see what can be ripped out. Often only a small subset of the possible branches is actually used; many others may have been tested once and abandoned."

4. Abstraction Depth (Can we abstract this?)

1. Relational database is a successful example of abstraction.
2. In ML it is difficult to come up with robust abstraction (how to create abstractions for a data stream, a model and a prediction?).
3. Example abstraction:
   Parameter-server abstraction - framework for distributed ML problems. "Data and workloads are distributed over worker nodes, while the server nodes maintain globally shared parameters, represented as dense or sparse vectors and matrices."

5. Common Smells (Something's fishy...)

1. "Code Smells are patterns of code that suggest there might be a problem, that there might be a better way of writing the code or that more design perhaps should go into it." (source)
2. Plain-Old-Data Type Smells.
   Check what flows in and out of ML systems. The data (signal, features, information, etc.) flowing in and out of ML systems is usually of type integer or float. "In a robust system, a model parameter should know if it is a log-odds multiplier or a decision threshold, and a prediction should know various pieces of information about the model that produced it and how it should be consumed."
3. Multiple-Language Smell.
   Using multiple programming languages, no matter the great packages written in it, makes it difficult to efficiently test and, later, transfer the ownership to others.
4. Prototype Smell.
   Constantly using prototype environments is an indication that the production system is "brittle, difficult to change, or could benefit from improved abstractions and interfaces".
   This leads to 2 potential problems:
   - Usage of the prototype environment as the production environment due to time pressures.
   - Prototype (small scale) solutions rarely reflect the reality of full-scale systems.

Configuration Debt

ML systems come with various configuration options:
1.  Choice of input features
2.  Algorithm-specific learning configurations (hyperparameters, number of nodes, layers, etc.)
3.  Pre- & post-processing methods
4.  Verification methods

In full-scale systems, configuration code far exceeds the number of lines of traditional code. Mistakes in configuration can lead to loss of time, waste of computing resources, and other production issues. Thus, verifying and testing configurations is crucial.

Example of difficult-to-handle configurations (Choice of input features):
1.  Feature A was incorrectly logged from 9/13-9/17.
2.  Feature B isn't available before 10/7.
3.  Feature C has to have 2 different acquisition methods due to changes in logging.
4.  Feature D isn't available in production. Substitute feature D' must be used.
5.  Feature Z requires extra training memory for the model to train efficiently.
6.  Feature Q has high latency, so it makes Feature R also unusable.

Configurations must be:
1.  Easy to change. It should seem as making a small change to the previous configuration.
2.  Difficult to make manual errors, omissions, or oversights.
3.  Easy to analyze and compare with the previous version.
4.  Easy to check (basic facts and details).
5.  Easy to detect unused or redundant settings.
6.  Code reviewed and maintained in a repository.

Dealing with the Changes in the External World

ML systems often directly interact with the ever-changing real world. Due to volatility in the real world, ML systems require continuous maintenance.

1. Fixed Thresholds in Dynamic Systems

1. For problems that predict whether a given sample is true or not it is required to set a decision threshold.
2. The classic approach is to choose a decision threshold that maximizes a certain metric (precision, recall, etc.).
3. When a model is retrained on a new data, the previous decision threshold usually becomes invalid.
4. Manually calculating thresholds is time consuming, so automatic optimization method is required.
5. Optimal approach is to extract a held-out validation set and use it to automatically calculate the optimal thresholds.
6. Approach used by the Google Data Scientists can be found in the "Unofficial Google Data Science" blog post.
7. My simple approach dealing with decision thresholds - blog post.

2. Monitoring and Testing

1. Unit tests and end-to-end tests of systems are valuable. However, in the real-world, these tests are simply not enough.
2. Realtime monitoring with automated response to issues is essential for long-term system reliability.
3. What should be monitored:
   a) Prediction Bias
   Is the distribution of predicted and observed labels equal? Although this test is not enough to detect a model that simply outputs average values of label occurrences without regard to the input features, it is surprisingly effective.
   For example:
   - Real-world behavior changes and, now, the training data distributions are not reflective of reality. In this case, analyzing predictions for various dimensions (e.g. based on race, gender, age, etc.) can isolate biases. Further, we can setup automated alerting in case we detect prediction bias.
   
   b) Action Limits
   In real-world ML systems that take actions such as bidding on items, classifying messages as spam, it is useful to set action limits as a sanity check. "If the system hits a limit for a given action, automated alerts should fire and trigger manual intervention or investigation."
   For example:
   - Setting maximum number of bids per hour
   - Setting maximum ratio of spams to regular emails to be 3/5
   
   c) Up-Stream Producers
   When data comes from up-stream producers, the up-stream producers need to be monitored, tested, and routinely meet objectives that take into account the downstream ML system. Most importantly, any issues in the up-stream must be propagated to the downstream ML system. And the ML system must also notify its downstream consumers.
   
4. Issues that occur in real-time and have impact on the system should be dealt with automatic measures. Human intervention can work, but if the issue is time-sensitive it won't work. Automatic measures are worth the investment.

Other Areas of ML-related Debt

1. Data Testing Debt

Compared to traditional systems, ML systems replace code with data. And if the input data is the core component, it makes sense to test the data. Basic sanity checks, as well as complex distribution checks are useful.

2. Reproducibility Debt

Reproducibility of experiments is an important principle of the scientific method. However, randomized algorithms, parallel learning and complex real-world interactions make it difficult to reproduce results.

3. Process Management Debt

Mature systems may have hundreds of models running in parallel. Updating configurations of many similar models safely and automatically, managing and assigning resources among models with different business properties, visualizing and detecting issues in the flow of data, and being able to recover the system from production issues are important features of a reliable ML system.

4. Cultural Debt

To keep an ML system healthy, create teams with diverse strengths in ML research and engineering.
Team culture should reward:
- Deletion of features
- Reduction of complexity
- Improvements in reproducibility
- Improvements in stability
- Monitoring as well as improvements in accuracy.

Conclusion

1. Speed is not evidence of low technical debt or good practices because the real cost of debt becomes apparent over time.
2. Useful questions for detecting ML technical debt:
   - How easily can an entirely new algorithmic approach be tested at full scale?
   - What is the transitive closure of all data dependencies? Transitive closure gives you the set of all places you can get to, from any starting place.
   - How precisely can the impact of a new change to the system be measured?
   - Does improving one model or signal degrade others?
   - How quickly can new members of the team be brought up to speed?
3. Key areas that will help reduce technical debt:
   - Maintainable ML
   - Better abstractions
   - Testing methodologies
   - Design patterns
4. Engineers and researchers both need to be aware of technical debt. "Research solutions that provide a tiny accuracy benefit at the cost of massive increases in system complexity are rarely wise practice."
5. Dealing with technical debt can only be achieved by a change in team culture. "Recognizing, prioritizing, and rewarding this effort is important for the long term health of successful ML teams."

And this is it!

In this post, I have tried to summarize the key points of the prominent paper "Hidden Technical Debt in Machine Learning Systems" by Google.