Dynamic treatment regime

In medical research, a dynamic treatment regime (DTR), adaptive intervention, or adaptive treatment strategy is a set of rules for choosing effective treatments for individual patients. Historically, medical research and the practice of medicine tended to rely on an acute care model for the treatment of all medical problems, including chronic illness. Treatment choices made for a particular patient under a dynamic regime are based on that individual's characteristics and history, with the goal of optimizing his or her long-term clinical outcome. A dynamic treatment regime is analogous to a policy in the field of reinforcement learning, and analogous to a controller in control theory. While most work on dynamic treatment regimes has been done in the context of medicine, the same ideas apply to time-varying policies in other fields, such as education, marketing, and economics.

See also

• Personalized medicine
• Reinforcement learning
• Q learning
• Optimal control
• Multi-armed bandit

Further reading

• Diaz, Francisco J.; Cogollo, Myladis R.; Spina, Edoardo; Santoro, Vincenza; Rendon, Diego M.; Leon, jose de (2012), "Drug Dosage Individualization Based on a Random-Effects Linear Model", Journal of Biopharmaceutical Statistics, 22 (3): 463–484, doi:10.1080/10543406.2010.547264, PMID 22416835
• Diaz, Francisco J.; Yeh, Hung-Weh; Leon, Jose de (2012), "Role of Statistical Random-Effects Linear Models in Personalized Medicine", Current Pharmacogenomics and Personalized Medicine, 10 (1): 22–32, doi:10.2174/1875692111201010022, PMC 3580802, PMID 23467392
• Banerjee, A.; Tsiatis, A. A. (2006), "Adaptive two-stage designs in phase II clinical trials", Statistics in Medicine, 25 (19): 3382–3395, doi:10.1002/sim.2501, PMID 16479547
• Collins, L. M.; Murphy, S. A.; Nair, V.; Strecher, V. (2005), "A strategy for optimizing and evaluating behavioral interventions", Annals of Behavioral Medicine, 30 (1): 65–73, doi:10.1207/s15324796abm3001_8, PMID 16097907
• Guo, X.; Tsiatis, A. A. (2005), "Estimation of survival distributions in two-stage randomization designs with censored data", International Journal of Biostatistics, 1 (1), doi:10.2202/1557-4679.1000
• Hernán, Miguel A.; Lanoy, Emilie; Costagliola, Dominique; Robins, James M. (2006), "Comparison of Dynamic Treatment Regimes via Inverse Probability Weighting", Basic & Clinical Pharmacology & Toxicology, 98 (3): 237–242, doi:10.1111/j.1742-7843.2006.pto_329.x, PMID 16611197
• Lavori, P. W.; Dawson, R. (2000), "A design for testing clinical strategies: biased adaptive within-subject randomization", Journal of the Royal Statistical Society, Series A, 163: 29–38, doi:10.1111/1467-985x.00154
• Lavori, P.W.; Rush, A.J.; Wisniewski, S.R.; Alpert, J.; Fava, M.; Kupfer, D.J.; Nierenberg, A.; Quitkin, F.M.; Sacheim, H.A.; Thase, M.E.; Trivedi, M (2001), "Strengthening clinical effectiveness trials: Equipoise-stratified randomization", Biological Psychiatry, 50 (10): 792–801, doi:10.1016/s0006-3223(01)01223-9, PMID 11720698
• Lavori, P. W.; Dawson, R (2003), "Dynamic treatment regimes: practical design considerations", Clinical Trials, 1 (1): 9–20, doi:10.1191/1740774s04cn002oa, PMID 16281458
• Lizotte, D. L.; Bowling, M.; Murphy, S. A. (2010), "Efficient Reinforcement Learning with Multiple Reward Functions for Randomized Clinical Trial Analysis" (PDF), Twenty-Seventh Annual International Conference on Machine Learning
• Lokhnygina, Y; Tsiatis, A. A. (2008), "Optimal two-stage group sequential designs", Journal of Statistical Planning and Inference, 138 (2): 489–499, doi:10.1016/j.jspi.2007.06.011
• Lunceford, J. K.; Davidian, M.; Tsiatis, A. A. (2002), "Estimation of survival distributions of treatment policies in two-stage randomization designs in clinical trials", Biometrics, 58 (1): 48–57, doi:10.1111/j.0006-341x.2002.00048.x, PMID 11890326
• Moodie, E. E. M.; Richardson, T. S.; Stephens, D. A. (2007), "Demystifying optimal dynamic treatment regimes", Biometrics, 63 (2): 447–455, doi:10.1111/j.1541-0420.2006.00686.x, PMID 17688497
• Murphy, Susan A.; van der Laan, M. J.; Robins, James M.; CPPRG (2001), "Marginal Mean Models for Dynamic Regimes", Journal of the American Statistical Association, 96 (456): 1410–1423, doi:10.1198/016214501753382327, PMC 2794446, PMID 20019887
• Murphy, Susan A. (2003), "Optimal Dynamic Treatment Regimes", Journal of the Royal Statistical Society, Series B, 65 (2): 331–366, doi:10.1111/1467-9868.00389, hdl:2027.42/74095
• Murphy, Susan A. (2005), "An Experimental Design for the Development of Adaptive Treatment Strategies" (PDF), Statistics in Medicine, 24 (10): 1455–1481, doi:10.1002/sim.2022, hdl:2027.42/39201, PMID 15586395
• Murphy, Susan A.; Daniel Almiral (2008), "Dynamic Treatment Regimes", Encyclopedia of Medical Decision Making: #
• Nair, V.; Strecher, V.; Fagerlin, A.; Ubel, P.; Resnicow, K.; Murphy, S.; Little, R.; Chakraborty, B.; Zhang, A. (2008), "Screening Experiments and Fractional Factorial Designs in Behavioral Intervention Research", American Journal of Public Health, 98 (8): 1354–1359, doi:10.2105/ajph.2007.127563, PMC 2446451, PMID 18556602
• Orellana, Liliana; Rotnitzky, Andrea; Robins, James M. (2010), "Dynamic Regime Marginal Structural Mean Models for Estimation of Optimal Dynamic Treatment Regimes, Part I: Main Content", The International Journal of Biostatistics, 6 (2), doi:10.2202/1557-4679.1200, hdl:10536/DRO/DU:30069855, archived from the original on 2011-06-11, retrieved 2010-04-12
• Orellana, Liliana; Rotnitzky, Andrea; Robins, James M. (2010), "Dynamic Regime Marginal Structural Mean Models for Estimation of Optimal Dynamic Treatment Regimes, Part II: Proofs of Results", The International Journal of Biostatistics, 6 (2): 9, doi:10.2202/1557-4679.1242, PMC 2854089, PMID 20405047, archived from the original on 2011-06-11, retrieved 2010-04-12
• Robins, James M. (2004), "Optimal structural nested models for optimal sequential decisions", in Lin, D. Y.; Heagerty, P. J. (eds.), Proceedings of the Second Seattle Symposium on Biostatistics, Springer, New York, pp. 189–326
• Robins, James M. (1986), "A new approach to causal inference in mortality studies with sustained exposure periods-application to control of the healthy worker survivor effect", Computers and Mathematics with Applications, 14: 1393–1512
• Robins, James M. (1987), "Addendum to 'A new approach to causal inference in mortality studies with sustained exposure periods-application to control of the healthy worker survivor effect'", Computers and Mathematics with Applications, 14 (9–12): 923–945, doi:10.1016/0898-1221(87)90238-0
• Rush, A.J.; Trivedi, M.; Fava, M. (2003), "Depression IV: STAR*D treatment trial for depression", American Journal of Psychiatry, 160 (2): 237, doi:10.1176/appi.ajp.160.2.237, PMID 12562566
• Schneider, L.S.; Tariot, P.N.; Lyketsos, C.G.; Dagerman, K.S.; Davis, K.L.; Davis, S.; Hsiao, J.K.; Jeste, D.V.; Katz, I.R.; Olin, J.T.; Pollock, B.G.; Rabins, P.V.; Rosenheck, R.A.; Small, G.W.; Lebowitz, B.; Lieberman, J.A. (2001), "National Institute of Mental Health clinical antipsychotic trials of intervention effectiveness (CATIE) Alzheimer disease trial methodology", American Journal of Geriatric Psychiatry, 9 (4): 346–360, doi:10.1097/00019442-200111000-00004, PMID 11739062
• Sutton, R. S.; Barto, A. G. (1998), Reinforcement Learning: An Introduction, MIT Press, ISBN 978-0-262-19398-6, archived from the original on 2009-09-04
• van der Laan, M. J.; Robins, James M. (2003), Unified Methods for Censored Longitudinal Data and Causality, Springer-Verlag, ISBN 978-0-387-95556-8
• van der Laan, M. J.; Petersen, M. L. (2004), History-Adjusted Marginal Structural Models and Statically-Optimal Dynamic Treatment Regimes
• Wagner, E. H.; Austin, B. T.; Davis, C.; Hindmarsh, M.; Schaefer, J.; Bonomi, A. (2001), "Improving Chronic Illness Care: Translating Evidence Into Action", Health Affairs, 20 (6): 64–78, doi:10.1377/hlthaff.20.6.64, PMID 11816692
• Wahed, A.. S.; Tsiatis, A. A. (2004), "Optimal estimator for the survival distribution and related quantities for treatment policies in two-stage randomization designs in clinical trials", Biometrics, 60 (1): 124–133, doi:10.1111/j.0006-341X.2004.00160.x, PMID 15032782
• Watkins, C. J. C. H. (1989), "Learning from Delayed Rewards", PhD thesis, Cambridge University, Cambridge, England
• Zhao, Y.; Kosorok, M. R.; Zeng, D. (2009), "Reinforcement learning design for cancer clinical trials", Statistics in Medicine, 28 (26): 3294–3315, doi:10.1002/sim.3720, PMC 2767418, PMID 19750510
• Zajonc, T. (2010), Bayesian Inference for Dynamic Treatment Regimes: Mobility, Equity, and Efficiency in Student Tracking, SSRN 1689707