~evilnick/juju-core/docs-redesign

</nav><a href="https://juju.ubuntu.com/" class="logo-ubuntu"><img src="https://juju.ubuntu.com/wp-content/themes/juju-website/img/logo-ubuntu.png"></a>

</div>

<h1>Resources</h1>

<h2>A collection of some of the most important online references for Juju users and developers.</h2>

</div>

</header>

<h1>User Guide</h1>

<ul>

<li class=""><a href="getting-started.html">Getting Started</a></li>

<li class=" sub"><a href="getting-started.html#intro">Introduction</a></li>

<li class=" sub"><a href="getting-started.html#install">Installation</a></li>

<li class=" sub"><a href="config-aws.html">AWS Configuration</a></li>

<li class=" sub"><a href="config-hpcloud.html">HP Cloud Configuration</a></li>

<li class=" sub"><a href="">OpenStack Configuration</a></li>

<li class=" sub"><a href="config-maas.html">MAAS Configuration</a></li>

<li class=" sub"><a href="getting-started.html#test">Testing your setup</a></li>

<li class=""><a href="charms.html">Using Charms</a></li>

<li class=" sub"><a href="charms.html#intro">What are Charms?</a></li>

<li class=" sub"><a href="charms-deploying.html">Deploying Services</a></li>

<li class=" sub"><a href="charms-constraints.html">Using constraints</a></li>

<li class=" sub"><a href="charms-config.html">Service Configuration</a></li>

<li class=" sub"><a href="charms-relations.html">Service Relationships</a></li>

<li class=" sub"><a href="charms-exposing.html">Exposing Services</a></li>

<li class=" sub"><a href="charms-scaling.html">Scaling Services</a></li>

<li class=" sub"><a href="charms-destroy.html">Destroying Services</a></li>

</ul>

<h1>Charm Authors</h1>

<li class=""><a href="authors-charm-writing.html">Writing a charm</a></li>

<li class=" sub"><a href="authors-subordinate-services.html">Subordinate services</a></li>

<li class=" sub"><a href="authors-implicit-relations.html">Implicit Relations</a></li>

<li class=" sub"><a href="authors-testing.html">Charm Testing</a></li>

<li class=" sub"><a href="authors-hooks.html">Hook debugging</a></li>

<li class=""><a href="authors-charm-store.html">The Juju Charm Store</a></li>

<li class=" sub"><a href="authors-charm-policy.html">Charm store policy</a></li>

<li class=" sub"><a href="authors-charm-best-practice.html">Best practices</a></li>

<li class=" sub"><a href="authors-charm-quality.html">Charm Quality Rating</a></li>

<h1>Reference</h1>

<ul>

<li class="sub"><a href="commands.html">Juju commands</a></li>

<li class="sub"><a href="glossary.html">Glossary</a></li>

<li class="sub"><a href="contributing.html">Contribute to the Docs!</a></li>

</ul>

100

</nav>

101

</div>

102

103

104

105

<h1>Charm Testing</h1>

106

<p>Juju has been designed from the start to foster a large collection of "charms". Charms are expected to number in the thousands, and be self contained, with well defined interfaces for defining their relationships to one another.</p>

107

<p>Because this is a large complex system, not unlike a Linux software distribution, there is a need to test the charms and how they interact with one another. This specification defines a plan for implementing a simple framework to help this happen.</p>

108

<p>Static tests have already been implemented in the <tt class="docutils literal"><span class="pre">charm</span> <span class="pre">proof</span></tt> command as part of <tt class="docutils literal"><span class="pre">charm-tools</span></tt>. Any static testing of charms is beyond the scope of this specification.</p>

109

110

111

<h2>Phase 1 - Generic tests</h2>

112

<p>All charms share some of the same characteristics. They all have a yaml file called <tt class="docutils literal"><span class="pre">metadata.yaml</span></tt>, and when deployed, juju will always attempt to progress the state of the service from install to config to started. Because of this, all charms can be tested using the following algorithm:</p>

113

<pre>deploy charm

114

 while state != started

115

      if timeout is reached, FAIL

116

      if state == install_error, config_error, or start_error, FAIL

117

      if state == started, PASS</pre>

118

119

<p>Other generic tests may be identified, so a collection of generic tests should be the focus of an implementation.</p>

120

<p>Note that this requirement is already satisfied by Mark Mims' jenkins tester: <a class="reference external" href="http://charmtests.markmims.com/">http://charmtests.markmims.com/</a></p>

121

122

<h2>Phase 2 - Charm Specific tests</h2>

123

<p>Charm authors will have the best insight into whether or not a charm is working properly.</p>

124

<p>A simple structure will be utilized to attach tests to charms. Under the charm root directory, a sub-directory named 'tests' will be scanned by a test runner for executable files matching the glob <tt class="docutils literal"><span class="pre">*.test</span></tt>. These will be run in lexical order by the test runner, with a predictible environment. The tests can make the following assumptions:</p>

125

126

<li>A minimal install of the release of Ubuntu which the charm is targetted at will be available.</li>

127

<li>A version of juju is installed and available in the system path.</li>

128

<li>A juju environment with no services deployed inside it is already bootstrapped, and will be the default for command line usage.</li>

129

<li>The CWD is the <tt class="docutils literal"><span class="pre">tests</span></tt> directory off the charm root.</li>

130

<li>Full network access to deployed nodes will be allowed.</li>

131

<li>the bare name of any charm in arguments to juju will be resolved to a charm url and/or repository arguments of the test runner's choice. This means that if you need mysql, you do not do <tt class="docutils literal"><span class="pre">juju</span> <span class="pre">deploy</span> <span class="pre">cs:mysql</span></tt> or <tt class="docutils literal"><span class="pre">juju</span> <span class="pre">deploy</span> <span class="pre">--repository</span> <span class="pre">~/charms</span> <span class="pre">local:mysql</span></tt>, but just <tt class="docutils literal"><span class="pre">juju</span> <span class="pre">deploy</span> <span class="pre">mysql</span></tt>. A wrapper will resolve this to the latest version of the given charm from the list of official charms.</li>

132

</ul>

133

<p>The following restrictions may be enforced:</p>

134

135

<li>Internet access will be restricted from the testing host.</li>

136

</ul>

137

<p>If present, tests/tests.yaml will be read to determine packages that need to be installed on the host running tests in order to facilitate the tests. The packages can <em>only</em> be installed from the official, default Ubuntu archive for the release which the charm is intended for, from any of the repositories enabled by default in said release. The format of tests.yaml is as such:</p>

138

<pre>packages: [ package1, package2, package3 ]</pre>

139

140

<p>If a tool is needed to perform a test and is not available in the Ubuntu archive, it can also be included in the <tt class="docutils literal"><span class="pre">tests/</span></tt> directory, as long as the file which contains it does not end in <tt class="docutils literal"><span class="pre">.test</span></tt>. Note that build tools cannot be assumed to be available on the testing system.</p>

141

142

<h3>Purpose of tests</h3>

143

<p>The purpose of these tests is to assert that the charm works well on the intended platform and performs the expected configuration steps. Examples of things to test in each charm beyond install/start is:</p>

144

145

<li>After install, expose, and adding of required relations, the service is listening on the intended ports and is functional.</li>

146

<li>Adding, removing, and re-adding a relation should work without error.</li>

147

<li>Setting config values should result in the config value reflected in the service's configuraion.</li>

148

<li>Adding multiple units to a web app charm and relating to a load balancer results in the same HTML on both units directly and the load balancer.</li>

149

</ul>

150

151

<h3>Exit Codes</h3>

152

<p>Upon exit, the test's exit code will be evaluated to mean the following:</p>

153

154

<li>0: Test passed</li>

155

<li>1: Failed test</li>

156

<li>100: Test is skipped because of incomplete environment</li>

157

</ul>

158

159

<h3>Output</h3>

160

<p>There is a general convention which output should follow, though it will not be interpreted by machine. On stdout, a message indicating the reason for the exit code should be printed, with a prefix string corresponding to the exit codes defined above. The correlation is:</p>

161

162

163

164

165

</ul>

166

<p>Anything else intentional should be prefixed with the word 'INFO'. If the

167

contents of files are to be logged, the contents should be preceeded by

168

<tt class="docutils literal"><span class="pre">INFO:</span> <span class="pre">BEGIN</span> <span class="pre">filename</span></tt>, where filename is a logical name unique to

169

this run of the test, and then the file ended with <tt class="docutils literal"><span class="pre">INFO:</span> <span class="pre">END</span> <span class="pre">filename</span></tt>.</p>

170

171

<h3>Example Tests<</h3>

172

173

<h4>Deploy requirements and Poll</h4>

174

<p>The test below <a class="footnote-reference" href="#id2" id="id1">[*]</a> deploys mediawiki with mysql and memcached related to it,

175

and then tests to make sure it returns a page via http with "<title>"

176

somewhere in the content.:</p>

177

<pre>#!/bin/sh

178

179

 set -e

180

181

 teardown() {

182

         if [ -n "$datadir" ] ; then

183

             if [ -f $datadir/passed ]; then

184

                 rm -r $datadir

185

             else

186

                 echo INFO: $datadir preserved

187

                 if [ -f $datadir/wget.log ] ; then

188

                     echo INFO: BEGIN wget.log

189

                     cat $datadir/wget.log

190

                     echo INFO: END wget.log

191

192

193

194

 }

195

 trap teardown EXIT

196

197

198

 juju deploy mediawiki

199

 juju deploy mysql

200

 juju deploy memcached

201

 juju add-relation mediawiki:db mysql:db

202

 juju add-relation memcached mediawiki

203

 juju expose mediawiki

204

205

 for try in `seq 1 600` ; do

206

         host=`juju status | tests/get-unit-info mediawiki public-address`

207

         if [ -z "$host" ] ; then

208

             sleep 1

209

         else

210

             break

211

212

 done

213

214

 if [ -z "$host" ] ; then

215

         echo FAIL: status timed out

216

         exit 1

217

218

219

 datadir=`mktemp -d ${TMPDIR:-/tmp}/wget.test.XXXXXXX`

220

 echo INFO: datadir=$datadir

221

222

 wget --tries=100 --timeout=6 http://$host/ -O - -a $datadir/wget.log | grep -q '<title>'

223

224

 if [ $try -eq 600 ] ; then

225

         echo FAIL: Timed out waiting.

226

         exit 1

227

228

229

 touch $datadir/passed

230

231

 trap - EXIT

232

 teardown

233

234

 echo PASS

235

 exit 0</pre>

236

237

<h3>Test config settings</h4>

238

<p>The following example tests checks to see if the default_port change the admin asks for is actually respected post-deploy:</p>

239

<pre>#!/bin/sh

240

241

 if [ -z "`which nc`" ] ; then

242

         echo "SKIP: cannot run tests without netcat"

243

         exit 100

244

245

246

 set -e

247

248

 juju deploy mongodb

249

 juju expose mongodb

250

251

 for try in `seq 1 600` ; do

252

        host=`juju status | tests/get-unit-info mongodb public-address`

253

        if [ -z "$host" ] ; then

254

            sleep 1

255

        else

256

            break

257

258

 done

259

260

 if [ -z "$host" ] ; then

261

        echo FAIL: status timed out

262

        exit 1

263

264

265

 assert_is_listening() {

266

        local port=$1

267

        listening=""

268

        for try in `seq 1 10` ; do

269

            if ! nc $host $port < /dev/null ; then

270

                continue

271

272

            listening="$port"

273

            break

274

        done

275

276

        if [ -z "$listening" ] ; then

277

           echo "FAIL: not listening on port $port after 10 retries"

278

           return 1

279

        else

280

           echo "PASS: listening on port $listening"

281

           return 0

282

283

 }

284

285

 assert_is_listening 27017

286

287

 juju set mongodb default_port=55555

288

289

 assert_is_listening 55555

290

 echo PASS: config change tests passed.

291

 exit 0</pre>

292

293

294

295

296

<tr><td class="label"><a class="fn-backref" href="#id1">[*]</a></td><td>get-unit-info

297

The example tests script uses a tool that is not widely available yet, <tt class="docutils literal"><span class="pre">get-unit-info</span></tt>. In the future enhancements should be made to juju core to allow such things to be made into plugins. Until then, it can be included in each test dir that uses it, or we can build a package of tools that are common to tests.</td></tr>

298

</tbody>

299

</table>

300

301

<h2>Test Runner</h2>

302

<p>A test runner will periodically poll the collection of charms for changes since the last test run. If there have been changes, the entire set of changes will be tested as one delta. This delta will be recorded in the test results in such a way where a developer can repeat the exact set of changes for debugging purposes.</p>

303

<p>All of the charms will be scanned for tests in lexical order by series, charm name, branch name. Non official charms which have not been reviewed by charmers will not have their tests run until the test runner's restrictions have been vetted for security, since we will be running potentially malicious code. It is left to the implementor to determine what mix of juju, client platform, and environment settings are appropriate, as all of these are variables that will affect the running charms, and so may affect the outcome.</p>

304

<p>If tests exit with services still in the environment, the test runner may clean them up, whether by destroying the environment or destroying the services explicitly, and the machines may be terminated as well. Any artifacts needed from the test machines should be retrieved and displayed before the test exits.</p>

305

</section>

306

</article>

307

</div>

308

</div>

309

</section>

310

311

312

<div>

313

314

<ul>

315

<li><a href="https://juju.ubuntu.com/get-started">Get started</a></li>

316

<li class="page_item"><a href="https://juju.ubuntu.com/get-started/local/">Local</a></li>

317

<li class="page_item"><a href="https://juju.ubuntu.com/get-started/amazon/">Amazon Web Services</a></li>

318

<li class="page_item"><a href="https://juju.ubuntu.com/get-started/hp-cloud/">HP Cloud</a></li>

319

<li class="page_item"><a href="https://juju.ubuntu.com/get-started/rackspace/">Rackspace</a></li>

320

<li class="page_item"><a href="https://juju.ubuntu.com/get-started/openstack/">Openstack</a></li>

321

322

</ul>

323

<ul>

324

<li><a href="https://juju.ubuntu.com/resources">Resources</a></li>

325

<li><a href="https://juju.ubuntu.com/docs">Documentation</a></li>

326

<li><a href="https://juju.ubuntu.com/resources/videos">Videos</a></li>

327

328

</ul>

329

<ul>

330

<li><a href="https://juju.ubuntu.com/community">Community</a></li>

331

332

<li class="page_item"><a href="https://juju.ubuntu.com/community/weekly-charm-meeting/">Charm Meetings</a></li>

333

<li class="page_item"><a href="https://juju.ubuntu.com/community/charmers/">Charmers</a></li>

334

<li class="page_item"><a href="https://lists.ubuntu.com/mailman/listinfo/juju">Mailing List</a></li>

335

336

337

</ul>

338

339

340

341

<li><a href="https://launchpad.net/charms">Charms</a></li>

342

</ul>

343

</nav>

344

</div>

345

</footer>

346

347

348

349

</body>

350

</html>

b'\\ No newline at end of file'

Older »