Modern problems in statistics often include estimators of high computational complexity and with complicated distributions. Statistical inference on such estimators usually relies on asymptotic normality assumptions, however, such assumptions are often not applicable for available sample sizes, due to dependencies in the data. A common alternative is the use of resampling procedures, such as bootstrapping, but these may be computationally intensive to an extent that renders them impractical for modern problems. In this article, we develop a method for fast construction of test-inversion bootstrap confidence intervals. Our approach uses quantile regression to model the quantile of an estimator conditional on the true value of the parameter. We apply this to the Watterson estimator of mutation rate in a standard coalescent model. We demonstrate an improved efficiency of up to 40% from using quantile regression compared to state of the art methods based on stochastic approximation, as measured by the number of simulations required to achieve comparable accuracy. Supplementary materials for this article are available online.