python-testing/05-pytest.html at gh-pages · swcarpentry/python-testing · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <meta name="generator" content="pandoc">
    <title>Software Carpentry: Testing</title>
    <link rel="shortcut icon" type="image/x-icon" href="/favicon.ico" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <link rel="stylesheet" type="text/css" href="css/bootstrap/bootstrap.css" />
    <link rel="stylesheet" type="text/css" href="css/bootstrap/bootstrap-theme.css" />
    <link rel="stylesheet" type="text/css" href="css/swc.css" />
    <link rel="alternate" type="application/rss+xml" title="Software Carpentry Blog" href="http://software-carpentry.org/feed.xml"/>
    <meta charset="UTF-8" />
    <!-- HTML5 shim, for IE6-8 support of HTML5 elements -->
    <!--[if lt IE 9]>
      <script src="http://html5shim.googlecode.com/svn/trunk/html5.js"></script>
    <![endif]-->
  </head>
  <body class="lesson">
    <div class="container card">
      <div class="banner">
        <a href="http://software-carpentry.org" title="Software Carpentry">
          <img alt="Software Carpentry banner" src="img/software-carpentry-banner.png" />
        </a>
      </div>
      <article>
      <div class="row">
        <div class="col-md-10 col-md-offset-1">
                    <a href="index.html"><h1 class="title">Testing</h1></a>
          <h2 class="subtitle">Running Tests with pytest</h2>
          <section class="objectives panel panel-warning">
<div class="panel-heading">
<h2 id="learning-objectives"><span class="glyphicon glyphicon-certificate"></span>Learning Objectives</h2>
</div>
<div class="panel-body">
<ul>
<li>Understand how to run a test suite using the pytest framework</li>
<li>Understand how to read the output of a pytest test suite</li>
</ul>
</div>
</section>
<p>We created a suite of tests for our mean function, but it was annoying to run them one at a time. It would be a lot better if there were some way to run them all at once, just reporting which tests fail and which succeed.</p>
<p>Thankfully, that exists. Recall our tests:</p>
<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="im">from</span> mean <span class="im">import</span> <span class="op">*</span>

<span class="kw">def</span> test_ints():
    num_list <span class="op">=</span> [<span class="dv">1</span>,<span class="dv">2</span>,<span class="dv">3</span>,<span class="dv">4</span>,<span class="dv">5</span>]
    obs <span class="op">=</span> mean(num_list)
    exp <span class="op">=</span> <span class="dv">3</span>
    <span class="cf">assert</span> obs <span class="op">==</span> exp

<span class="kw">def</span> test_zero():
    num_list<span class="op">=</span>[<span class="dv">0</span>,<span class="dv">2</span>,<span class="dv">4</span>,<span class="dv">6</span>]
    obs <span class="op">=</span> mean(num_list)
    exp <span class="op">=</span> <span class="dv">3</span>
    <span class="cf">assert</span> obs <span class="op">==</span> exp

<span class="kw">def</span> test_double():
    <span class="co"># This one will fail in Python 2</span>
    num_list<span class="op">=</span>[<span class="dv">1</span>,<span class="dv">2</span>,<span class="dv">3</span>,<span class="dv">4</span>]
    obs <span class="op">=</span> mean(num_list)
    exp <span class="op">=</span> <span class="fl">2.5</span>
    <span class="cf">assert</span> obs <span class="op">==</span> exp

<span class="kw">def</span> test_long():
    big <span class="op">=</span> <span class="dv">100000000</span>
    obs <span class="op">=</span> mean(<span class="bu">range</span>(<span class="dv">1</span>,big))
    exp <span class="op">=</span> big<span class="op">/</span><span class="fl">2.0</span>
    <span class="cf">assert</span> obs <span class="op">==</span> exp

<span class="kw">def</span> test_complex():
    <span class="co"># given that complex numbers are an unordered field</span>
    <span class="co"># the arithmetic mean of complex numbers is meaningless</span>
    num_list <span class="op">=</span> [<span class="dv">2</span> <span class="op">+</span> 3j, <span class="dv">3</span> <span class="op">+</span> 4j, <span class="op">-</span><span class="dv">32</span> <span class="op">-</span> 2j]
    obs <span class="op">=</span> mean(num_list)
    exp <span class="op">=</span> <span class="va">NotImplemented</span>
    <span class="cf">assert</span> obs <span class="op">==</span> exp</code></pre></div>
<p>Once these tests are written in a file called <code>test_mean.py</code>, the command <code>py.test</code> can be called from the directory containing the tests:</p>
<div class="sourceCode"><pre class="sourceCode bash"><code class="sourceCode bash">$ <span class="kw">py.test</span></code></pre></div>
<pre class="output"><code>collected 5 items

test_mean.py ....F

================================== FAILURES ===================================
________________________________ test_complex _________________________________

    def test_complex():
        # given that complex numbers are an unordered field
        # the arithmetic mean of complex numbers is meaningless
        num_list = [2 + 3j, 3 + 4j, -32 - 2j]
        obs = mean(num_list)
        exp = NotImplemented
&gt;       assert obs == exp
E       assert (-9+1.6666666666666667j) == NotImplemented

test_mean.py:34: AssertionError
===================== 1 failed, 4 passed in 2.71 seconds ======================</code></pre>
<p>In the above case, the pytest package ‘sniffed-out’ the tests in the directory and ran them together to produce a report of the sum of the files and functions matching the regular expression <code>[Tt]est[-_]*</code>.</p>
<p>The major boon a testing framework provides is exactly that, a utility to find and run the tests automatically. With pytest, this is the command-line tool called <code>py.test</code>. When <code>py.test</code> is run, it will search all directories below where it was called, find all of the Python files in these directories whose names start or end with <code>test</code>, import them, and run all of the functions and classes whose names start with <code>test</code> or <code>Test</code>. This automatic registration of test code saves tons of human time and allows us to focus on what is important: writing more tests.</p>
<p>When you run <code>py.test</code>, it will print a dot (<code>.</code>) on the screen for every test that passes, an <code>F</code> for every test that fails or where there was an unexpected error. In rarer situations you may also see an <code>s</code> indicating a skipped tests (because the test is not applicable on your system) or a <code>x</code> for a known failure (because the developers could not fix it promptly). After the dots, pytest will print summary information.</p>
<section class="challenge panel panel-success">
<div class="panel-heading">
<h2 id="fix-the-failing-code"><span class="glyphicon glyphicon-pencil"></span>Fix The Failing Code</h2>
</div>
<div class="panel-body">
<p>Without changing the tests, alter the mean.py file from the previous section until it passes. When it passes, <code>py.test</code> will produce results like the following:</p>
<div class="sourceCode"><pre class="sourceCode bash"><code class="sourceCode bash">$ <span class="kw">py.test</span></code></pre></div>
<p><sub>~</sub> {.output} collected 5 items</p>
<p>test_mean.py …..</p>
<p>========================== 5 passed in 2.68 seconds ===========================</p>
</div>
</section>
<p>As we write more code, we would write more tests, and pytest would produce more dots. Each passing test is a small, satisfying reward for having written quality scientific software. Now that you know how to write tests, let’s go into what can go wrong.</p>
        </div>
      </div>
      </article>
      <div class="footer">
        <a class="label swc-blue-bg" href="http://software-carpentry.org">Software Carpentry</a>
        <a class="label swc-blue-bg" href="https://github.com/swcarpentry/lesson-template">Source</a>
        <a class="label swc-blue-bg" href="mailto:admin@software-carpentry.org">Contact</a>
        <a class="label swc-blue-bg" href="LICENSE.html">License</a>
      </div>
    </div>
    <!-- Javascript placed at the end of the document so the pages load faster -->
    <script src="http://software-carpentry.org/v5/js/jquery-1.9.1.min.js"></script>
    <script src="css/bootstrap/bootstrap-js/bootstrap.js"></script>
  </body>
</html>