Publications
2024
On Average Optimality for Non-Stationary Markov Decision Processes in Borel Spaces
Guo, X., Huang, Y., & Zhang, Y. (n.d.). On Average Optimality for Non-Stationary Markov Decision Processes in Borel Spaces. Mathematics of Operations Research. doi:10.1287/moor.2023.0169
On the Continuity of the Projection Mapping from Strategic Measures to Occupation Measures in Absorbing Markov Decision Processes
Piunovskiy, A., & Zhang, Y. (2024). On the Continuity of the Projection Mapping from Strategic Measures to Occupation Measures in Absorbing Markov Decision Processes. Applied Mathematics & Optimization, 89(3). doi:10.1007/s00245-024-10124-7
Extreme occupation measures in Markov decision processes with an absorbing state.
Piunovskiy, A., & Zhang, Y. (2024). Extreme occupation measures in Markov decision processes with an absorbing state.. SIAM Journal on Control and Optimization.
2023
On the structure of optimal solutions in a mathematical programming problem in a convex space
Piunovskiy, A., & Zhang, Y. (2023). On the structure of optimal solutions in a mathematical programming problem in a convex space. Operations Research Letters, 51(5), 488-493. doi:10.1016/j.orl.2023.07.006
2022
Gradual-impulsive control for continuous-time Markov decision processes with total undiscounted costs and constraints: linear programming approach via a reduction method
Piunovskiy, A., & Zhang, Y. (2022). Gradual-impulsive control for continuous-time Markov decision processes with total undiscounted costs and constraints: linear programming approach via a reduction method. SIAM Journal on Control and Optimization.
2021
Aggregated occupation measures and linear programming approach to constrained impulse control problems
Piunovskiy, A., & Zhang, Y. (2021). Aggregated occupation measures and linear programming approach to constrained impulse control problems. JOURNAL OF MATHEMATICAL ANALYSIS AND APPLICATIONS, 499(2). doi:10.1016/j.jmaa.2021.125070
Modern Trends in Controlled Stochastic Processes: Theory and Applications, V.III
Piunovskiy, A., & Zhang, Y. (Eds.) (2021). Modern Trends in Controlled Stochastic Processes:. Springer International Publishing. doi:10.1007/978-3-030-76928-4
On Finite Approximations to Markov Decision Processes with Recursive and Nonlinear Discounting
Deng, F., Guo, X., & Zhang, Y. (2021). On Finite Approximations to Markov Decision Processes with Recursive and Nonlinear Discounting. In Emergence, Complexity and Computation (pp. 221-247). Springer International Publishing. doi:10.1007/978-3-030-76928-4_11
ON GRADUAL-IMPULSE CONTROL OF CONTINUOUS-TIME MARKOV DECISION PROCESSES WITH EXPONENTIAL UTILITY
Guo, X., Kurushima, A., Piunovskiy, A., & Zhang, Y. (2021). ON GRADUAL-IMPULSE CONTROL OF CONTINUOUS-TIME MARKOV DECISION PROCESSES WITH EXPONENTIAL UTILITY. ADVANCES IN APPLIED PROBABILITY, 53(2), 301-334. doi:10.1017/apr.2020.64
Linear programming approach to optimal impulse control problems with functional constraints
Piunovskiy, A., & Zhang, Y. (2021). Linear programming approach to optimal impulse control problems with functional constraints. Journal of Mathematical Analysis and Applications, 124817. doi:10.1016/j.jmaa.2020.124817
A useful technique for piecewise deterministic Markov decision processes
Zhang, Y., & Guo, X. (2021). A useful technique for piecewise deterministic Markov decision processes. Operations Research Letters. doi:10.1016/j.orl.2020.11.002
2020
On Risk-Sensitive Piecewise Deterministic Markov Decision Processes
Guo, X., & Zhang, Y. (2020). On Risk-Sensitive Piecewise Deterministic Markov Decision Processes. APPLIED MATHEMATICS AND OPTIMIZATION, 81(3), 685-710. doi:10.1007/s00245-018-9485-x
On reducing a constrained gradual-impulsive control problem for a jump Markov model to a model with gradual control only
Piunovskiy, A., & Zhang, Y. (2020). On reducing a constrained gradual-impulsive control problem for a jump Markov model to a model with gradual control only. SIAM Journal on Control and Optimization, 58(1), 192-214. doi:10.1137/19M1248303
Continuous-Time Markov Decision Processes
Piunovskiy, A., & Zhang, Y. (2020). Continuous-Time Markov Decision Processes. Springer International Publishing. doi:10.1007/978-3-030-54987-9
2019
Optimal impulse control of dynamical systems
Piunovskiy, A. B., Plakhov, A., Torres, D., & Zhang, Y. (2019). Optimal impulse control of dynamical systems. SIAM Journal on Control and Optimization, 57(4), 2720-2752. doi:10.1137/18M1212069
Finite horizon risk-sensitive continuous-time Markov decision processes with unbounded transition and cost rates
Guo, X., Liu, Q., & Zhang, Y. (2019). Finite horizon risk-sensitive continuous-time Markov decision processes with unbounded transition and cost rates. 4OR. doi:10.1007/s10288-019-0398-6
2018
Hitting Times in Markov Chains with Restart and their Application to Network Centrality
Avrachenkov, K., Piunovskiy, A., & Zhang, Y. (2018). Hitting Times in Markov Chains with Restart and their Application to Network Centrality. METHODOLOGY AND COMPUTING IN APPLIED PROBABILITY, 20(4), 1173-1188. doi:10.1007/s11009-017-9600-5
Impulsive Control for G-AIMD Dynamics with Relaxed and Hard Constraints.
Avrachenkov, K., Piunovskiy, A. B., & Zhang, Y. (2018). Impulsive Control for G-AIMD Dynamics with Relaxed and Hard Constraints.. In CDC (pp. 880-887). IEEE. Retrieved from https://doi.org/10.1109/CDC37347.2018
On the Nonexplosion and Explosion for Nonhomogeneous Markov Pure Jump Processes
Zhang, Y. (2018). On the Nonexplosion and Explosion for Nonhomogeneous Markov Pure Jump Processes. JOURNAL OF THEORETICAL PROBABILITY, 31(3), 1322-1355. doi:10.1007/s10959-017-0763-3
NOWAK'S THEOREM ON PROBABILITY MEASURES INDUCED BY STRATEGIES REVISITED
Kurushima, A., Piunovskiy, A., & Zhang, Y. (2018). NOWAK'S THEOREM ON PROBABILITY MEASURES INDUCED BY STRATEGIES REVISITED. THEORY OF PROBABILITY AND ITS APPLICATIONS, 62(2), 328-334. doi:10.1134/S0040585X97T988630
2017
NOTE ON DISCOUNTED CONTINUOUS-TIME MARKOV DECISION PROCESSES WITH A LOWER BOUNDING FUNCTION
Guo, X., Piunovskiy, A., & Zhang, Y. (2017). NOTE ON DISCOUNTED CONTINUOUS-TIME MARKOV DECISION PROCESSES WITH A LOWER BOUNDING FUNCTION. JOURNAL OF APPLIED PROBABILITY, 54(4), 1071-1088. doi:10.1017/jpr.2017.53
Continuous-Time Markov Decision Processes with Exponential Utility
Zhang, Y. (2017). Continuous-Time Markov Decision Processes with Exponential Utility. SIAM JOURNAL ON CONTROL AND OPTIMIZATION, 55(4), 2636-2660. doi:10.1137/16M1086261
Zero-sum continuous-time Markov pure jump game over a fixed duration
Guo, X., & Zhang, Y. (2017). Zero-sum continuous-time Markov pure jump game over a fixed duration. Journal of Mathematical Analysis and Applications, 452(2), 1194-1208. doi:10.1016/j.jmaa.2017.03.073
Constrained Continuous-Time Markov Decision Processes on the Finite Horizon
Guo, X., Huang, Y., & Zhang, Y. (2017). Constrained Continuous-Time Markov Decision Processes on the Finite Horizon. APPLIED MATHEMATICS AND OPTIMIZATION, 75(2), 317-341. doi:10.1007/s00245-016-9352-6
Constrained total undiscounted continuous-time Markov decision processes
Guo, X., & Zhang, Y. (2017). Constrained total undiscounted continuous-time Markov decision processes. Bernoulli, 23(3), 1694-1736. doi:10.3150/15-BEJ793
2016
Optimality of mixed policies for average continuous-time Markov decision processes with constraints
Guo, X., & Zhang, Y. (2016). Optimality of mixed policies for average continuous-time Markov decision processes with constraints. Mathematics of Operations Research, 41(4), 1161-1207. doi:10.1287/moor.2015.0777
2015
Infinite horizon optimal impulsive control with applications to Internet congestion control
Avrachenkov, K., Habachi, O., Piunovskiy, A., & Zhang, Y. (2015). Infinite horizon optimal impulsive control with applications to Internet congestion control. INTERNATIONAL JOURNAL OF CONTROL, 88(4), 703-716. doi:10.1080/00207179.2014.971436
ON THE FIRST PASSAGE <i>g</i>-MEAN-VARIANCE OPTIMALITY FOR DISCOUNTED CONTINUOUS-TIME MARKOV DECISION PROCESSES
Guo, X., Huang, X., & Zhang, Y. (2015). ON THE FIRST PASSAGE <i>g</i>-MEAN-VARIANCE OPTIMALITY FOR DISCOUNTED CONTINUOUS-TIME MARKOV DECISION PROCESSES. SIAM JOURNAL ON CONTROL AND OPTIMIZATION, 53(3), 1406-1424. doi:10.1137/140968872
2014
AVERAGE OPTIMALITY FOR CONTINUOUS-TIME MARKOV DECISION PROCESSES UNDER WEAK CONTINUITY CONDITIONS
Zhang, Y. (2014). AVERAGE OPTIMALITY FOR CONTINUOUS-TIME MARKOV DECISION PROCESSES UNDER WEAK CONTINUITY CONDITIONS. JOURNAL OF APPLIED PROBABILITY, 51(4), 954-970. Retrieved from https://www.webofscience.com/
Average Optimality for Continuous-Time Markov Decision Processes Under Weak Continuity Conditions
Zhang, Y. (2014). Average Optimality for Continuous-Time Markov Decision Processes Under Weak Continuity Conditions. Journal of Applied Probability, 51(04), 954-970. doi:10.1017/s0021900200011918
Discounted continuous-time Markov decision processes with unbounded rates and randomized history-dependent policies: the dynamic programming approach
Piunovskiy, A., & Zhang, Y. (2014). Discounted continuous-time Markov decision processes with unbounded rates and randomized history-dependent policies: the dynamic programming approach. 4OR, 12(1), 49-75. doi:10.1007/s10288-013-0236-1
First Passage Optimality for Continuous-Time Markov Decision Processes With Varying Discount Factors and History-Dependent Policies
Guo, X., Song, X., & Zhang, Y. (2014). First Passage Optimality for Continuous-Time Markov Decision Processes With Varying Discount Factors and History-Dependent Policies. IEEE TRANSACTIONS ON AUTOMATIC CONTROL, 59(1), 163-174. doi:10.1109/TAC.2013.2281475
Markov decision processes with iterated coherent risk measures
Chu, S., & Zhang, Y. (2014). Markov decision processes with iterated coherent risk measures. INTERNATIONAL JOURNAL OF CONTROL, 87(11), 2286-2293. doi:10.1080/00207179.2014.909947
2013
Markov Processes with Restart
Avrachenkov, K., Piunovskiy, A., & Zhang, Y. (2013). Markov Processes with Restart. Journal of Applied Probability, 50(04), 960-968. doi:10.1017/s0021900200013735
Markov Processes with Restart
Avrachenkov, K., Piunovskiy, A., & Zhang, Y. (2013). Markov Processes with Restart. Journal of Applied Probability, 50(4), 960-968. doi:10.1239/jap/1389370093
Absorbing Continuous-Time Markov Decision Processes with Total Cost Criteria
Guo, X., Vykertas, M., & Zhang, Y. (2013). Absorbing Continuous-Time Markov Decision Processes with Total Cost Criteria. Advances in Applied Probability, 45(2), 490-519. doi:10.1239/aap/1370870127
Convex analytic approach to constrained discounted Markov decision processes with non-constant discount factors
Zhang, Y. (2013). Convex analytic approach to constrained discounted Markov decision processes with non-constant discount factors. TOP, 21(2), 378-408. doi:10.1007/s11750-011-0186-8
2012
Fluid Approximations to Markov Decision Processes with Local Transitions
Piunovskiy, A., & Zhang, Y. (2012). Fluid Approximations to Markov Decision Processes with Local Transitions. In Systems & Control: Foundations & Applications (pp. 225-238). Birkhäuser Boston. doi:10.1007/978-0-8176-8337-5_13
The Transformation Method for Continuous-Time Markov Decision Processes
Piunovskiy, A., & Zhang, Y. (2012). The Transformation Method for Continuous-Time Markov Decision Processes. JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS, 154(2), 691-712. doi:10.1007/s10957-012-0015-8
2011
DISCOUNTED CONTINUOUS-TIME MARKOV DECISION PROCESSES WITH UNBOUNDED RATES: THE CONVEX ANALYTIC APPROACH
Piunovskiy, A., & Zhang, Y. (2011). DISCOUNTED CONTINUOUS-TIME MARKOV DECISION PROCESSES WITH UNBOUNDED RATES: THE CONVEX ANALYTIC APPROACH. SIAM JOURNAL ON CONTROL AND OPTIMIZATION, 49(5), 2032-2061. doi:10.1137/10081366X
Accuracy of fluid approximations to controlled birth-and-death processes: absorbing case
Piunovskiy, A., & Zhang, Y. (2011). Accuracy of fluid approximations to controlled birth-and-death processes: absorbing case. MATHEMATICAL METHODS OF OPERATIONS RESEARCH, 73(2), 159-187. doi:10.1007/s00186-010-0340-3
ASYMPTOTIC FLUID OPTIMALITY AND EFFICIENCY OF THE TRACKING POLICY FOR BANDWIDTH-SHARING NETWORKS
Avrachenkov, K., Piunovskiy, A., & Zhang, Y. (2011). ASYMPTOTIC FLUID OPTIMALITY AND EFFICIENCY OF THE TRACKING POLICY FOR BANDWIDTH-SHARING NETWORKS. JOURNAL OF APPLIED PROBABILITY, 48(1), 90-113. doi:10.1239/jap/1300198138
2010
Convergence of trajectories and optimal buffer sizing for MIMD congestion control
Zhang, Y., Piunovskiy, A., Ayesta, U., & Avrachenkov, K. (2010). Convergence of trajectories and optimal buffer sizing for MIMD congestion control. Computer Communications, 33(2), 149-159. doi:10.1016/j.comcom.2009.08.008
2008
Fluid Model of an Internet Router under the MIMD Control Scheme
Ayesta, U., Piunovskiy, A., & Zhang, Y. (2008). Fluid Model of an Internet Router under the MIMD Control Scheme. In Telecommunications Modeling, Policy, and Technology (pp. 239-251 (13 pages)). NY: Springer.
Fluid model of an Internet router under the MIMD control scheme
Ayesta, U., Piunovskiy, A. B., & Zhang, Y. (2008). Fluid model of an Internet router under the MIMD control scheme. TELECOMMUNICATIONS MODELING, POLICY, AND TECHNOLOGY, 239-+. Retrieved from https://www.webofscience.com/