← Back to branch summary

~bigdata-dev/charms/trusty/apache-hadoop-plugin/unstable

« back to all changes in this revision

Viewing changes to README.md

  • Committer: Cory Johns
  • Date: 2015-06-08 21:32:27 UTC
  • mfrom: (95.1.1 apache-hadoop-plugin)
  • Revision ID: cory.johns@canonical.com-20150608213227-7o6vcj23lgofjljt
Added benchmarking

Show diffs side-by-side

added added

removed removed

Lines of Context:
README.md
25
25
    juju deploy cs:~bigdata-dev/apache-pig pig
26
26
    juju add-relation plugin pig
27
27
 
 
28
## Benchmarking
 
29
 
 
30
    You can perform a terasort benchmark, in order to gauge performance of your environment:
 
31
 
 
32
        $ juju action do plugin/0 terasort
 
33
        Action queued with id: cbd981e8-3400-4c8f-8df1-c39c55a7eae6
 
34
        $ juju action fetch --wait 0 cbd981e8-3400-4c8f-8df1-c39c55a7eae6
 
35
        results:
 
36
          meta:
 
37
            composite:
 
38
              direction: asc
 
39
              units: ms
 
40
              value: "206676"
 
41
          results:
 
42
            raw: '{"Total vcore-seconds taken by all map tasks": "439783", "Spilled Records":
 
43
              "30000000", "WRONG_LENGTH": "0", "Reduce output records": "10000000", "HDFS:
 
44
              Number of bytes read": "1000001024", "Total vcore-seconds taken by all reduce
 
45
              tasks": "50275", "Reduce input groups": "10000000", "Shuffled Maps ": "8", "FILE:
 
46
              Number of bytes written": "3128977482", "Input split bytes": "1024", "Total
 
47
              time spent by all reduce tasks (ms)": "50275", "FILE: Number of large read operations":
 
48
              "0", "Bytes Read": "1000000000", "Virtual memory (bytes) snapshot": "7688794112",
 
49
              "Launched map tasks": "8", "GC time elapsed (ms)": "11656", "Bytes Written":
 
50
              "1000000000", "FILE: Number of read operations": "0", "HDFS: Number of write
 
51
              operations": "2", "Total megabyte-seconds taken by all reduce tasks": "51481600",
 
52
              "Combine output records": "0", "HDFS: Number of bytes written": "1000000000",
 
53
              "Total time spent by all map tasks (ms)": "439783", "Map output records": "10000000",
 
54
              "Physical memory (bytes) snapshot": "2329722880", "FILE: Number of write operations":
 
55
              "0", "Launched reduce tasks": "1", "Reduce input records": "10000000", "Total
 
56
              megabyte-seconds taken by all map tasks": "450337792", "WRONG_REDUCE": "0",
 
57
              "HDFS: Number of read operations": "27", "Reduce shuffle bytes": "1040000048",
 
58
              "Map input records": "10000000", "Map output materialized bytes": "1040000048",
 
59
              "CPU time spent (ms)": "195020", "Merged Map outputs": "8", "FILE: Number of
 
60
              bytes read": "2080000144", "Failed Shuffles": "0", "Total time spent by all
 
61
              maps in occupied slots (ms)": "439783", "WRONG_MAP": "0", "BAD_ID": "0", "Rack-local
 
62
              map tasks": "2", "IO_ERROR": "0", "Combine input records": "0", "Map output
 
63
              bytes": "1020000000", "CONNECTION": "0", "HDFS: Number of large read operations":
 
64
              "0", "Total committed heap usage (bytes)": "1755840512", "Data-local map tasks":
 
65
              "6", "Total time spent by all reduces in occupied slots (ms)": "50275"}'
 
66
        status: completed
 
67
        timing:
 
68
          completed: 2015-05-28 20:55:50 +0000 UTC
 
69
          enqueued: 2015-05-28 20:53:41 +0000 UTC
 
70
          started: 2015-05-28 20:53:44 +0000 UTC
 
71
 
28
72
 
29
73
## Deploying in Network-Restricted Environments
30
74