Papers by Timothy Brathwaite
arXiv (Cornell University), Apr 22, 2024
Deep neural networks (DNNs) have been increasingly applied in travel demand modeling because of t... more Deep neural networks (DNNs) have been increasingly applied in travel demand modeling because of their automatic feature learning, high predictive performance, and economic interpretability. Nevertheless, DNNs frequently present behaviorally irregular patterns, significantly limiting their practical potentials and theoretical validity in travel behavior modeling. This study proposes strong and weak behavioral regularities as novel metrics to evaluate the monotonicity of individual demand functions (known as the "law of demand"), and further designs a constrained optimization framework with six gradient regularizers to enhance DNNs' behavioral regularity. The empirical benefits of this framework are illustrated by applying these regularizers to travel survey data from Chicago and London, which enables us to examine the trade-off between predictive power and behavioral regularity for large versus small sample scenarios and in-domain versus out-of-domain generalizations. The results demonstrate that, unlike models with strong behavioral foundations such as the multinomial logit, the benchmark DNNs cannot guarantee behavioral regularity. However, after applying gradient regularization, we increase DNNs' behavioral regularity by around 6 percentage points while retaining their relatively high predictive power. In the small sample scenario, gradient regularization is more effective than in the large sample scenario, simultaneously improving behavioral regularity by about 20 percentage points and loglikelihood by around 1.7%. Compared with the in-domain generalization of DNNs, gradient regularization works more effectively in out-of-domain generalization: it drastically improves the behavioral regularity of poorly performing benchmark DNNs by around 65 percentage points, highlighting the criticality of behavioral regularization for improving model transferability and applications in forecasting. Moreover, the proposed optimization framework is applicable to other neural network-based choice models such as TasteNets. Future studies could use behavioral regularity as a metric along with log-likelihood, prediction accuracy, and F1 score when evaluating travel demand models, and investigate other methods to further enhance behavioral regularity when adopting complex machine learning models.
arXiv (Cornell University), Jun 6, 2018
Typically, discrete choice modelers develop ever-more advanced models and estimation methods. Com... more Typically, discrete choice modelers develop ever-more advanced models and estimation methods. Compared to the impressive progress in model development and estimation, model-checking techniques have lagged behind. Often, choice modelers use only crude methods to assess how well an estimated model represents reality. Such methods usually stop at checking parameter signs, model elasticities, and ratios of model coefficients. In this paper, I greatly expand the discrete choice modelers' assessment toolkit by introducing model checking procedures based on graphical displays of predictive simulations. Overall, my contributions are as follows. Methodologically, I introduce a general and 'semi-automatic' algorithm for checking discrete choice models via predictive simulations. By combining new graphical displays with existing plots, I introduce methods for checking one's data against one's model in terms of the model's predicted distributions of choices (P (Y)), choices given explanatory variables (P (Y | X)), and explanatory variables given choices (P (X | Y)). Empirically, I demonstrate my proposed methods by checking the models from Brownstone and Train (1998). Through this case study, I show that my proposed methods can point out lack-of-model-fit in one's models and suggest concrete model improvements that substantively change the results of one's policy analysis. Moreover, the case study highlights a practical trade-off between precision and robustness in model checking.
arXiv (Cornell University), Nov 13, 2017
In the 1960's, the logistic regression model from statistics and the binary probit model from psy... more In the 1960's, the logistic regression model from statistics and the binary probit model from psychology were linked with random utility theory, thereby connecting such methods with economic theory. Since then, the fields of statistics, computer science, and machine learning have created numerous methods for modeling discrete choices. However, these newer methods have not been derived from or linked with economic theories of human decision making. We believe this lack of economic interpretation is one reason discrete choice modelers have been slow to adopt these newer methods. Our paper begins bridging this gap by providing a microeconomic framework for decision trees: a popular machine learning method. Specifically, we show how decision trees represent a non-compensatory decision protocol known as disjunctions-of-conjunctions and how this protocol generalizes many of the noncompensatory rules used in the discrete choice literature so far. Additionally, we show how existing decision tree variants address many economic concerns that choice modelers might have. Beyond theoretical interpretations, we contribute to the existing literature of two-stage, semi-compensatory modeling and to the existing decision tree literature. In particular, we formulate the first bayesian model tree, thereby allowing for uncertainty in the estimated non-compensatory rules as well as for context-dependent preference heterogeneity in one's second-stage choice model. Using an application of bicycle mode choice in the San Francisco Bay Area, we estimate our bayesian model tree, and we find that it is over 1,000 times more likely to be closer to the true data-generating process than a multinomial logit model (MNL). Qualitatively, our bayesian model tree automatically finds the effect of bicycle infrastructure investment to be moderated by travel distance, socio-demographics and topography, and our model identifies diminishing returns from bicycle lane investments. These qualitative differences lead the bayesian model trees to produce forecasts that directly align with the observed bicycle mode shares in regions with abundant bicycle infrastructure such as Davis, CA and the Netherlands. In comparison, the forecasts of the MNL model are overly optimistic.
Every day, decision-makers make choices among finite and discrete sets of alternatives. For examp... more Every day, decision-makers make choices among finite and discrete sets of alternatives. For example, people decide whether to walk, bike, take transit, or drive to work; shoppers decide which of the available brands of toothpaste to buy; and firms decide which vacant buildings they will rent for office space. Across these disparate domains, discrete choice models mathematically represent the procedures that analysts believe decision-makers are using to make such choices. Historically, the field of discrete choice modeling grew mainly out of economics, and this lineage has had long-lasting methodological ramifications. In particular, despite the great mathematical similarity between discrete choice models and models in statistics, machine learning, and causal inference, discrete choice research remains mostly siloed, seldom drawing from or contributing to methods in these related disciplines. In this dissertation, we help demolish the methodological silo around discrete choice research. Drawing from recent techniques in statistics, machine learning, and causal inference, we remove substantive limitations on the decision-making processes that could be be represented and predicted with previously available discrete choice methods. At the same time, by addressing concerns of discrete choice modelers, we make methodological contributions to the fields of statistics and machine learning, and we identify future research areas where discrete choice modelers are well suited to advancing the state of the art in causal inference. Importantly, the methodological advances described above were not divorced from today's societal concerns. Given that more and more government agencies are (unsuccessfully) attempting to raise bicycle commuting rates in their jurisdictions, we guide our interactions with the statistics, machine learning, and causal inference literatures by trying to more accurately model an individual's choice of commuting by bicycle. In particular, we use parametric link functions from statistics to better model the adoption and abandonment of bicycling. From machine learning, we use decision trees to represent the non-compensatory decision protocols that individuals appear to follow when deciding whether to commute by bicycle, i To Carla and Keturah for their steadfast love and support, and to Andrew and Darren for teaching me to ride without fear. ii Contents Contents ii List of Figures iv List of Tables v This dissertation was made possible by the love, support, encouragement, teaching, and guidance of a vast community of people-only a handful of whom I can acknowledge here. Beginning with this dissertation's focus, I'll start by acknowledging Andrew Malone and Darren Lee. Without you two, I may never have made bicycling such a central part of my identity, and this work would have felt much less personally meaningful. Likewise, thanks also to Frank Proulx, for demonstrating that a topic as planner-y as bicycling could be the subject of quantitative academic inquiry, even in engineering. Thanks for always being one step ahead in life, proofreading all of my papers, and paving the way for me overall. Scholastically, thanks are due first and foremost to my advisor, Professor Joan Walker. Thank you for giving me the space to grow academically and to explore whatever topics I was interested in, even when you suggested otherwise. Thanks for caring when others appeared not to and working to keep me enrolled. Thanks for having brilliant suggestions when I most needed them, for showing me "how to write in a way that it makes a difference," and for teaching me the importance of "motivating the problem." It has been a privilege to work with you and learn from you. Next, thanks to my committee members: Assistant Professors William Fithian and Alexei Pozdnoukhov. Thank you Will, for the refreshing enthusiasm with which you engaged my research and the time you spent talking through details with me. Alexei, thanks for always questioning the "established" interpretations of my models, for challenging me to think predictively, and for encouraging a healthy sense of skepticism. Lastly, thank you Akshay Vij for (without exaggeration) being the absolute best. Thanks for spending hours putting me on an academically expedient path, for showing me how discrete choice worked "from the code up," for talking me out of quitting, for sharing your struggles with me, and for being a shining exemplar of all I wanted to be. You made this dissertation intellectually possible. To UCCONNECT and Paul Waddell, thanks for funding me and making this dissertation financially possible. Socially, the list of those to thank is truly massive, and for those not mentioned in name, know that I am still so grateful for all your support. Thanks to: • Aaron Dolor for being a shoulder to lean on and a constant partner in this academic journey for over a decade now-from Brooklyn to the Bay. • Feras 'N'zali' El Zarwi and Sreeta Gorripaty for being my two complementary stooges and accompanying me on the crazy ride that was these past three (four?) years. • Jesùs Cuellar for being my company in despair, my brother from another mother, and the best friend one could ask for. • Darren Reger for being a constant source of laughter, and for seeing me at my absolute worst and still believing I had something to offer at my best. • Ari Takata-Vasquez and Miles Bianchi for giving me perspective on life, always being willing to grab a beer when I was down, and reminding me of the importance of sleep. 1 Note, Chapters 1, 2, and 6 use the pronoun "I" whereas Chapters 3, 4, and 5 use the pronoun "we." This discrepancy exists because Chapters 3-5 are based on work that was performed with collaborators, but Chapters 1, 2, and 6 represent writings that were produced without the vetting or consensus of collaborators. I thought it best not to wrongly associate any ideas that my collaborators might disagree with, so I have chosen to use "I" when I am not referring to joint work or opinions.
Journal of choice modelling, Mar 1, 2018
This paper is about the general disconnect that we see, both in practice and in literature, betwe... more This paper is about the general disconnect that we see, both in practice and in literature, between the disciplines of travel demand modeling and causal inference. In this paper, we assert that travel demand modeling should be one of the many fields that focuses on the production of valid causal inferences, and we hypothesize about reasons for the current disconnect between the two bodies of research. Furthermore, we explore the potential benefits of uniting these two disciplines. We consider what travel demand modeling can gain from greater incorporation of techniques and perspectives from the causal inference literatures, and we briefly discuss what the causal inference literature might gain from the work of travel demand modelers. In this paper, we do not attempt to "solve" issues related to the drawing of causal inferences from travel demand models. Instead, we hope to spark a larger discussion both within and between the travel demand modeling and causal inference literatures. In particular, we hope to incite discussion about the necessity of drawing causal inferences in travel demand applications and the methods by which one might credibly do so.
This paper is about the general disconnect that we see, both in practice and in literature, betwe... more This paper is about the general disconnect that we see, both in practice and in literature, between the disciplines of travel demand modeling and causal inference. In this paper, we assert that travel demand modeling should be one of the many fields that focuses on the production of valid causal inferences, and we hypothesize about reasons for the current disconnect between the two bodies of research. Furthermore, we explore the potential benefits of uniting these two disciplines. We consider what travel demand modeling can gain from greater incorporation of techniques and perspectives from the causal inference literatures, and we briefly discuss what the causal inference literature might gain from the work of travel demand modelers. In this paper, we do not attempt to "solve" issues related to the drawing of causal inferences from travel demand models. Instead, we hope to spark a larger discussion both within and between the travel demand modeling and causal inference lit...
Typically, discrete choice modelers develop ever-more advanced models and estimation methods. Com... more Typically, discrete choice modelers develop ever-more advanced models and estimation methods. Compared to the impressive progress in model development and estimation, model-checking techniques have lagged behind. Often, choice modelers use only crude methods to assess how well an estimated model represents reality. Such methods usually stop at checking parameter signs, model elasticities, and ratios of model coefficients. In this paper, I greatly expand the discrete choice modelers' assessment toolkit by introducing model checking procedures based on graphical displays of predictive simulations. Overall, my contributions are as follows. Methodologically, I introduce a general and 'semi-automatic' algorithm for checking discrete choice models via predictive simulations. By combining new graphical displays with existing plots, I introduce methods for checking one's data against one's model in terms of the model's predicted distributions of choices (P(Y)), choic...
arXiv: Applications, 2018
Typically, discrete choice modelers develop ever-more advanced models and estimation methods. Com... more Typically, discrete choice modelers develop ever-more advanced models and estimation methods. Compared to the impressive progress in model development and estimation, model-checking techniques have lagged behind. Often, choice modelers use only crude methods to assess how well an estimated model represents reality. Such methods usually stop at checking parameter signs, model elasticities, and ratios of model coefficients. In this paper, I greatly expand the discrete choice modelers' assessment toolkit by introducing model checking procedures based on graphical displays of predictive simulations. Overall, my contributions are as follows. Methodologically, I introduce a general and 'semi-automatic' algorithm for checking discrete choice models via predictive simulations. By combining new graphical displays with existing plots, I introduce methods for checking one's data against one's model in terms of the model's predicted distributions of P (Y), P (Y|X), and P...
Author(s): Brathwaite, Timothy | Advisor(s): Walker, Joan L | Abstract: Every day, decision-maker... more Author(s): Brathwaite, Timothy | Advisor(s): Walker, Joan L | Abstract: Every day, decision-makers make choices among finite and discrete sets of alternatives. For example, people decide whether to walk, bike, take transit, or drive to work; shoppers decide which of the available brands of toothpaste to buy; and firms decide which vacant buildings they will rent for office space. Across these disparate domains, discrete choice models mathematically represent the procedures that analysts believe decision-makers are using to make such choices.Historically, the field of discrete choice modeling grew mainly out of economics, and this lineage has had long-lasting methodological ramifications. In particular, despite the great mathematical similarity between discrete choice models and models in statistics, machine learning, and causal inference, discrete choice research remains mostly siloed, seldom drawing from or contributing to methods in these related disciplines.In this dissertation, ...
arXiv: Applications, 2017
We provide a microeconomic framework for decision trees: a popular machine learning method. Speci... more We provide a microeconomic framework for decision trees: a popular machine learning method. Specifically, we show how decision trees represent a non-compensatory decision protocol known as disjunctions-of-conjunctions and how this protocol generalizes many of the non-compensatory rules used in the discrete choice literature so far. Additionally, we show how existing decision tree variants address many economic concerns that choice modelers might have. Beyond theoretical interpretations, we contribute to the existing literature of two-stage, semi-compensatory modeling and to the existing decision tree literature. In particular, we formulate the first bayesian model tree, thereby allowing for uncertainty in the estimated non-compensatory rules as well as for context-dependent preference heterogeneity in one's second-stage choice model. Using an application of bicycle mode choice in the San Francisco Bay Area, we estimate our bayesian model tree, and we find that it is over 1,000 t...
Journal of Choice Modelling
This paper is about the general disconnect that we see, both in practice and in literature, betwe... more This paper is about the general disconnect that we see, both in practice and in literature, between the disciplines of travel demand modeling and causal inference. In this paper, we assert that travel demand modeling should be one of the many fields that focuses on the production of valid causal inferences, and we hypothesize about reasons for the current disconnect between the two bodies of research. Furthermore, we explore the potential benefits of uniting these two disciplines. We consider what travel demand modeling can gain from greater incorporation of techniques and perspectives from the causal inference literatures, and we briefly discuss what the causal inference literature might gain from the work of travel demand modelers. In this paper, we do not attempt to "solve" issues related to the drawing of causal inferences from travel demand models. Instead, we hope to spark a larger discussion both within and between the travel demand modeling and causal inference literatures. In particular, we hope to incite discussion about the necessity of drawing causal inferences in travel demand applications and the methods by which one might credibly do so.
Journal of Choice Modelling
Class imbalance, where there are great differences between the number of observations associated ... more Class imbalance, where there are great differences between the number of observations associated with particular discrete outcomes, is common within transportation and other fields. In the statistics literature, one explanation for class imbalance that has been hypothesized is an asymmetric (rather than the typically symmetric) choice probability function. Unfortunately, few relatively simple models exist for testing this hypothesis in transportation settings-settings that are inherently multinomial. Our paper fills this gap. In particular, we address the following questions: "how can one construct asymmetric, closed-form, finite-parameter models of multinomial choice" and "how do such models compare against commonly used symmetric models?" Methodologically, we introduce (1) a new class of closed-form, finite-parameter, multinomial choice models, (2) a procedure for using these models to extend existing binary choice models to the multinomial setting, and (3) a procedure for creating new binary choice models (both symmetric and asymmetric). Together, our contributions allow us to create new asymmetric, closed-form, finite-parameter multinomial choice models. We demonstrate our methods by developing four new asymmetric, multinomial choice models. Empirically, most of our models strongly dominate the multinomial logit (MNL) model in terms of in-sample and out-of-sample log-likelihoods. Moreover, analyzing two policy applications, we find practical differences between the MNL and our new asymmetric models. Our results suggest that while asymmetric models may not always outperform symmetric ones, asymmetric choice models are worth testing because they might have better statistical performance and entail substantively different policy and financial implications when compared with traditional symmetric models, such as the MNL.
Journal of Choice Modelling
Class imbalance, where there are great differences between the number of observations associated ... more Class imbalance, where there are great differences between the number of observations associated with particular discrete outcomes, is common within transportation and other fields. In the statistics literature, one explanation for class imbalance that has been hypothesized is an asymmetric (rather than the typically symmetric) choice probability function. Unfortunately, few relatively simple models exist for testing this hypothesis in transportation settings-settings that are inherently multinomial. Our paper fills this gap. In particular, we address the following questions: "how can one construct asymmetric, closed-form, finite-parameter models of multinomial choice" and "how do such models compare against commonly used symmetric models?" Methodologically, we introduce (1) a new class of closed-form, finite-parameter, multinomial choice models, (2) a procedure for using these models to extend existing binary choice models to the multinomial setting, and (3) a procedure for creating new binary choice models (both symmetric and asymmetric). Together, our contributions allow us to create new asymmetric, closed-form, finite-parameter multinomial choice models. We demonstrate our methods by developing four new asymmetric, multinomial choice models. Empirically, most of our models strongly dominate the multinomial logit (MNL) model in terms of in-sample and out-of-sample log-likelihoods. Moreover, analyzing two policy applications, we find practical differences between the MNL and our new asymmetric models. Our results suggest that while asymmetric models may not always outperform symmetric ones, asymmetric choice models are worth testing because they might have better statistical performance and entail substantively different policy and financial implications when compared with traditional symmetric models, such as the MNL.
Uploads
Papers by Timothy Brathwaite