Bootstrap and Resampling Software and Resources

Contents of this page:

  • Bootstrap/Resampling Software
  • Resampling Short Course
  • Bootstrap for Introductory Statistics
  • Articles and Technical Reports

    Bootstrap/Resampling Software

    S+Resample now ships with the latest version of S-PLUS. If you have an older version of S-PLUS, you may download S+Resample from http://www.insightful.com/downloads/libraries. If you have an older version of S-PLUS, see bestVersions.html.

    S+Resample includes:

  • Menu-driven-interface (Windows only). Easy access to common tasks.
  • General-purpose S-PLUS software, primarily upgrades of existing S-PLUS functions in order to support capabilities of bootstrap software. For example, many functions now have an optional "weights" argument.
  • New versions of `bootstrap' and related code, with added functionality or bug fixes.
  • New resampling functions, for permutation tests, cross validation, bootstrapping prediction errors, parametric and smoothed bootstrapping.
  • New functions for bootstrap tilting inferences and diagnostics.

    A feature of this package is bootstrap tilting, which generally provides very accurate (second-order accurate) confidence intervals, with better finite-sample performance than other bootstrap methods, and require only 1/17 or 1/37 as many bootstrap replications as other bootstrap methods (for 95% confidence intervals), e.g. 60 replications instead of 1000 or 2000.

    For more details on what is included see README.txt, ReleaseNotes.txt, and ChangeLog.txt. For a comparison of S+Resample and other resampling software available in S-PLUS and R, see comparison.txt. For validation routines (using do.test) see resampleLoop.zip.

    This project was supported by the National Institutes of Health under SBIR Phase-II grant 2R44CA67734--02 "Statistical Software for Resampling Methods", and by the National Science Foundation, under SBIR Phase-II grant DMI-9861360, "Bootstrap Tilting Inference and Large Data Sets".

    Contributors to this software include Andrea Borning, Steve Ellis, Chris Fraley, Tim Hesterberg, Shan Jin, Charles Roosen, James Schimert, and Robert Thurman. We also appreciate advice received from David S. Moore, Brad Efron, Art Owen, Luigi Salmaso, and Rob Tibshirani.

    This software is owned by Insightful; I no longer work there, but will answer some questions related to the software.

    Short Course: Bootstrap Methods and Permutation Tests

    This is an introduction to the bootstrap, permutation tests, and other resampling methods. For a course description and details see course.asp. I have given this course in various formats, ranging from a two-day hands-on course to half-day lecture-only, public or private, in Albuquerque, Boston, Chicago, Cincinnati, L.A., Little Rock, Minneapolis, Portland, Rochester MN, San Francisco, Washington D.C., Basel, Basingstoke UK, Bedford UK, London, Manchester, Montpellier FR, Toronto, and Zurich. Since I am no longer at Insightful I won't give this course as frequently. I do plan to give a version in Sept 2009 for the ASA's FDA/Industry Statistics Workshop. Contact me if you are interested in other courses.

    Bootstrap for Introductory Statistics

    The bootstrap and permutation tests offer ways to help students better understand concepts such as sampling distributions, standard errors, confidence intervals, and P-values. Bootstrap Methods and Permutation Tests (BMPT) by Hesterberg, Moore, Monaghan, Clipson, and Epstein was written as an introduction to these methods, with a focus on the pedagogical value. There are two versions of BMPT, written as supplemental chapters for two different books, but either can be used independently by people who want an introduction to bootstrap methods and permutation tests.

    The first version ("BMPT/PBS") is a supplemental chapter for The Practice of Business Statistics: Using Data for Decisions by Moore, McCabe, Duckworth and Sclove. This is available from W. H. Freeman, ISBN 0-7167-5726-5 for about $7, or is available at http://bcs.whfreeman.com/pbs/cat_160/PBS18.pdf.

    The second version ("BMPT/IPS") is a supplemental chapter for Introduction to the Practice of Statistics, 5th Edition by Moore and McCabe. This is available at http://bcs.whfreeman.com/ips5e/content/cat_080/pdf/moore14.pdf. See also http://www.whfreeman.com/ipsresample.

    S-PLUS data libraries and supplements for PBS and IPS

    There are S-PLUS libraries to accompany both versions, containing datasets, example scripts, and documentation.
    For BMPT/PBS download PBSdata.zip.
    For BMPT/IPS download IPSdata.zip.
    Unzip either library, then follow instructions in INSTALL.txt.

    For a general introduction to S-PLUS, see the S-PLUS Guide for Moore and McCabe's Introduction to the Practice of Statistics, Fifth Edition.

    Free Student version of S-PLUS

    You need S-PLUS to use either the IPSdata or PBSdata library. Students may get a free student version of S-PLUS from http://elms03.e-academy.com/splus. The e-academy procedures can be difficult to follow, you may find these instructions helpful: e-academy-instructions.txt.

    Teachers may get a free evaluation copy from http://www.insightful.com/contactus/request_cd.asp.

    Articles and Technical Reports related to this software:

  • Tim Hesterberg (2008) It's Time To Retire the "n >= 30" Rule, Proceedings of the American Statistical Association, Statistical Computing Section (CD-ROM).
  • Hesterberg, Tim C. (2008), It's Time To Retire "n >= 30", (talk at JSM08).
  • Hesterberg, Tim C. (2007), Bootstrap, in Wiley Encyclopedia of Clinical Trials, DOI: 10.1002/9780471462422.eoct392.
  • Hesterberg, Tim C. (2006), "Bootstrapping Students' Understanding of Statistical Concepts", in: Thinking and Reasoning with Data and Chance: 68th NCTM Yearbook (2006), Sixty-eighth Yearbook, National Council of Teachers of Mathematics, editors Gail F. Burrill and Portia C. Elliot, pages 391-416.
  • Laura M. Chihara, Gregory L. Snow, and Tim C. Hesterberg (2006), S-PLUS Guide for Moore's The Basic Practice of Statistics, Fourth Edition W. H. Freeman, N.Y.
  • Tim Hesterberg, David S. Moore, Shaun Monaghan, Ashley Clipson, and Rachel Epstein (2005), Bootstrap Methods and Permutation Tests, 2nd edition, W. H. Freeman, N.Y.
  • Gregory Snow, Laura Chihara, and Tim Hesterberg (2005), S-PLUS Guide for Moore and McCabe's Introduction to the Practice of Statistics, Fifth Edition W. H. Freeman, N.Y.
  • Hesterberg, Tim (2005), Resampling for Planning Clinical Trials-Using S+Resample, poster for "Statistical Methods in Biopharmacy" conference, Paris.
  • Hesterberg, Tim C. (2004), Unbiasing the Bootstrap-Bootknife Sampling vs. Smoothing, Proceedings of the Section on Statistics and the Environment, American Statistical Association, 2924-2930.
  • Tim Hesterberg, Shaun Monaghan, David S. Moore, Ashley Clipson, and Rachel Epstein (2003), Bootstrap Methods and Permutation Tests, W. H. Freeman, N.Y.
  • Hesterberg, Tim C. (2002), "Performance Evaluation using Fast Permutation Tests" Proceedings of the Tenth International Conference on Telecommunication Systems, 465-474.
  • Hesterberg, Tim C. (2001), "Bootstrap Tilting Diagnostics", Proceedings of the Statistical Computing Section (CD-ROM), American Statistical Association.
  • Hesterberg, Tim C. (1999), "Bootstrap Tilting Confidence Intervals and Hypothesis Tests", Computing Science and Statistics, 31, 389--393, Interface Foundation of North America, Fairfax Station, VA.
  • Hesterberg, Tim C. (1999), "Bootstrap Tilting Confidence Intervals", Technical Report No. 84, Research Department, MathSoft, Inc., 1700 Westlake Ave. N., Suite 500, Seattle, WA 98109.
  • Ellis, Stephen J. and Tim C. Hesterberg (1999), "Computation of Weighted Functional Statistics Using Software That Does Not Support Weights", Technical Report No. 85, Research Department, MathSoft, Inc., 1700 Westlake Ave. N., Suite 500, Seattle, WA 98109.
  • Hesterberg, Tim C. and Stephen J. Ellis (1999), "Linear Approximations for Functional Statistics in Large-Sample Applications", Technical Report No. 86, Research Department, MathSoft, Inc., 1700 Westlake Ave. N., Suite 500, Seattle, WA 98109.
  • Hesterberg, Tim C. (1999), "Smoothed bootstrap and jackboot sampling", Technical Report No. 87, Research Department, MathSoft, Inc., 1700 Westlake Ave. N., Suite 500, Seattle, WA 98109.
  • Hesterberg, Tim C. (1998), "Simulation and Bootstrapping for Teaching Statistics", Proceedings of the Statistical Education Section, American Statistical Association, 44--52.
  • Hesterberg, Tim C. (1998), "Bootstrap Tilting Inference and Large Data Sets", Proposal to NSF SBIR Program.
  • Hesterberg, Tim C. (1997), "Fast Bootstrapping by Combining Importance Sampling and Concomitants", Computing Science and Statistics, 29(2), 72-78. Interface Foundation of North America, Fairfax Station, VA. Eds E. J. Wegman and S. Azen.
  • Hesterberg, T. C. (1997), "The bootstrap and empirical likelihood", Proceedings of the Section on Statistical Computing, American Statistical Association, 34-36.
  • Hesterberg, Tim C. (1997), "Matched-Block Bootstrap for Long Memory Processes", Technical Report No. 66, Research Department, MathSoft, Inc. 1700 Westlake Ave. N., Suite 500, Seattle, WA 98109.

    For other articles (including references to published articles related to this software) see www.insightful.com/Hesterberg/articles


    Tim Hesterberg, +1 206-285-1625

    Updated 16 April 2009.