← Back to branch summary

~ubuntu-branches/ubuntu/maverick/eucalyptus/maverick

« back to all changes in this revision

Viewing changes to clc/modules/storage-controller/src/main/java/edu/ucsb/eucalyptus/storage/BlockStorageChecker.java

  • Committer: Bazaar Package Importer
  • Author(s): Dave Walker (Daviey)
  • Date: 2010-07-21 17:27:10 UTC
  • mfrom: (1.1.38 upstream)
  • Revision ID: james.westby@ubuntu.com-20100721172710-7xv07dmdqgivc3t9
Tags: 2.0~bzr1211-0ubuntu1
https://launchpad.net/bugs/597330
https://launchpad.net/bugs/595421
* New major upstream version merge, 2.0 (r1211).
* debian/patches/:
  - 01-wsdl-stubs.patch, debian/wsdl.md5sums: wsdl stubs updated.
  - 02-Makefile.patch: Updated to reflect new code layout.
  - 07-local_support_euca_conf-in.patch: Updated to reflect new code layout.
  - 08-ubuntu-default-networking.patch: Refreshed.
  - 09-small-128-192MB.patch: Updated to point to new location.
  - 10-disable-iscsi.patch: Refreshed.
  - 11-state-cleanup-memleakfix.patch: Removed, fixed upstream.
  - 15-fix-default-ramdisk.patch: Updated to point to new location.
  - 16-kvm_libvirt_xml_default_use_kvm.patch: Updated to reflect changes.
  - 17-fix_walrus_OOM_errors.patch: Removed, fixed upstream.
  - 18-priv_security.patch: Updated to reflect upstream changes.
  - 20-brute-force-webui.patch: Updated to reflect upstream changes. 
  - 21-eucalyptus-1.7-with-gwt-1.6.4.patch: New patch, allows 
    eucalyptus-1.7 to be built against gwt 1.6.4. Based on patch courtesy 
    of Dmitrii Zagorodnov, upstream. (LP: #597330)
* debian/eucalyptus-java-common.links: 
  - Changed symlink for groovy, point to groovy.all.jar, making compatiable 
    with groovy versions >1.7. (LP: #595421)
  - Added ant.jar & jetty-rewrite-handler.jar as they are now required.
* debian/control
  - & debian/build-jars: Added libjavassist-java and libjetty-extra-java as 
    build dependencies.
  - Added libjetty-extra-java as a dependency of eucalyptus-java-common
* The binary resulting jar's have been renamed from eucalyptus-*-1.6.2.jar
  to eucalyptus-*-main.jar:    
  - debian/eucalyptus-cc.upstart
  - debian/eucalyptus-cloud.install
  - debian/eucalyptus-common.eucalyptus.upstart
  - debian/eucalyptus-java-common.install
  - debian/eucalyptus-network.upstart
  - debian/eucalyptus-sc.install
  - debian/eucalyptus-walrus.install
* debian/eucalyptus-java-common.install: New upstream jars that have been
  installed:
  - eucalyptus-db-hsqldb-ext-main.jar
  - eucalyptus-component-main.jar
* debian/control:
  - Updated Standards Version to 3.8.4 (no change)
  - Updated the upstream Homepage to: http://open.eucalyptus.com/
  - Changed Vcs-Bzr to reflect new location of Ubuntu hosted development branch.
  - Made the Build Dependency of groovy and the binary eucalyptus-java-common
    package depend on version >=1.7.

Show diffs side-by-side

added added

removed removed

Lines of Context:
clc/modules/storage-controller/src/main/java/edu/ucsb/eucalyptus/storage/BlockStorageChecker.java
1
 
/*******************************************************************************
2
 
 *Copyright (c) 2009  Eucalyptus Systems, Inc.
3
 
 * 
4
 
 *  This program is free software: you can redistribute it and/or modify
5
 
 *  it under the terms of the GNU General Public License as published by
6
 
 *  the Free Software Foundation, only version 3 of the License.
7
 
 * 
8
 
 * 
9
 
 *  This file is distributed in the hope that it will be useful, but WITHOUT
10
 
 *  ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
11
 
 *  FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
12
 
 *  for more details.
13
 
 * 
14
 
 *  You should have received a copy of the GNU General Public License along
15
 
 *  with this program.  If not, see <http://www.gnu.org/licenses/>.
16
 
 * 
17
 
 *  Please contact Eucalyptus Systems, Inc., 130 Castilian
18
 
 *  Dr., Goleta, CA 93101 USA or visit <http://www.eucalyptus.com/licenses/>
19
 
 *  if you need additional information or have any questions.
20
 
 * 
21
 
 *  This file may incorporate work covered under the following copyright and
22
 
 *  permission notice:
23
 
 * 
24
 
 *    Software License Agreement (BSD License)
25
 
 * 
26
 
 *    Copyright (c) 2008, Regents of the University of California
27
 
 *    All rights reserved.
28
 
 * 
29
 
 *    Redistribution and use of this software in source and binary forms, with
30
 
 *    or without modification, are permitted provided that the following
31
 
 *    conditions are met:
32
 
 * 
33
 
 *      Redistributions of source code must retain the above copyright notice,
34
 
 *      this list of conditions and the following disclaimer.
35
 
 * 
36
 
 *      Redistributions in binary form must reproduce the above copyright
37
 
 *      notice, this list of conditions and the following disclaimer in the
38
 
 *      documentation and/or other materials provided with the distribution.
39
 
 * 
40
 
 *    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
41
 
 *    IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
42
 
 *    TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
43
 
 *    PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
44
 
 *    OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
45
 
 *    EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
46
 
 *    PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
47
 
 *    PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
48
 
 *    LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
49
 
 *    NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
50
 
 *    SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. USERS OF
51
 
 *    THIS SOFTWARE ACKNOWLEDGE THE POSSIBLE PRESENCE OF OTHER OPEN SOURCE
52
 
 *    LICENSED MATERIAL, COPYRIGHTED MATERIAL OR PATENTED MATERIAL IN THIS
53
 
 *    SOFTWARE, AND IF ANY SUCH MATERIAL IS DISCOVERED THE PARTY DISCOVERING
54
 
 *    IT MAY INFORM DR. RICH WOLSKI AT THE UNIVERSITY OF CALIFORNIA, SANTA
55
 
 *    BARBARA WHO WILL THEN ASCERTAIN THE MOST APPROPRIATE REMEDY, WHICH IN
56
 
 *    THE REGENTS’ DISCRETION MAY INCLUDE, WITHOUT LIMITATION, REPLACEMENT
57
 
 *    OF THE CODE SO IDENTIFIED, LICENSING OF THE CODE SO IDENTIFIED, OR
58
 
 *    WITHDRAWAL OF THE CODE CAPABILITY TO THE EXTENT NEEDED TO COMPLY WITH
59
 
 *    ANY SUCH LICENSES OR RIGHTS.
60
 
 *******************************************************************************/
61
 
/*
62
 
 *
63
 
 * Author: Sunil Soman sunils@cs.ucsb.edu
64
 
 */
65
 
 
66
 
package edu.ucsb.eucalyptus.storage;
67
 
 
68
 
import edu.ucsb.eucalyptus.cloud.entities.SnapshotInfo;
69
 
import edu.ucsb.eucalyptus.cloud.entities.VolumeInfo;
70
 
import edu.ucsb.eucalyptus.cloud.ws.BlockStorage;
71
 
import edu.ucsb.eucalyptus.cloud.ws.HttpWriter;
72
 
import edu.ucsb.eucalyptus.cloud.ws.SnapshotProgressCallback;
73
 
import edu.ucsb.eucalyptus.ic.StorageController;
74
 
 
75
 
import com.eucalyptus.util.EucalyptusCloudException;
76
 
import com.eucalyptus.util.StorageProperties;
77
 
import org.apache.commons.httpclient.HttpClient;
78
 
import org.apache.commons.httpclient.methods.GetMethod;
79
 
import org.apache.log4j.Logger;
80
 
 
81
 
import com.eucalyptus.util.EntityWrapper;
82
 
 
83
 
import java.io.File;
84
 
import java.net.URL;
85
 
import java.util.HashMap;
86
 
import java.util.List;
87
 
import java.util.Map;
88
 
import java.util.UUID;
89
 
 
90
 
public class BlockStorageChecker {
91
 
        private static Logger LOG = Logger.getLogger(BlockStorageChecker.class);
92
 
        private StorageManager volumeStorageManager;
93
 
        private StorageManager snapshotStorageManager;
94
 
        private LogicalStorageManager blockManager;
95
 
        private static boolean transferredPending = false;
96
 
 
97
 
        public BlockStorageChecker(StorageManager volumeStorageManager, 
98
 
                        StorageManager snapshotStorageManager, 
99
 
                        LogicalStorageManager blockManager) {
100
 
                this.volumeStorageManager = volumeStorageManager;
101
 
                this.snapshotStorageManager = snapshotStorageManager;
102
 
                this.blockManager = blockManager;
103
 
        }
104
 
 
105
 
        public void startupChecks() {
106
 
                new StartupChecker().start();
107
 
        }
108
 
 
109
 
        private class StartupChecker extends Thread {
110
 
                public StartupChecker() {}
111
 
 
112
 
                public void run() {
113
 
                        try {
114
 
                                cleanup();
115
 
                        } catch (EucalyptusCloudException e) {
116
 
                                LOG.error("Startup cleanup failed", e);
117
 
                        }
118
 
                        blockManager.startupChecks();
119
 
                }
120
 
        }
121
 
 
122
 
        public void cleanup() throws EucalyptusCloudException {
123
 
                cleanVolumes();
124
 
                cleanSnapshots();
125
 
        }
126
 
 
127
 
        public void cleanVolumes() {
128
 
                cleanStuckVolumes();
129
 
                cleanFailedVolumes();
130
 
        }
131
 
 
132
 
        public void cleanSnapshots() {
133
 
                cleanStuckSnapshots();
134
 
                cleanFailedSnapshots();
135
 
        }
136
 
 
137
 
        public void cleanStuckVolumes() {
138
 
                EntityWrapper<VolumeInfo> db = StorageController.getEntityWrapper();
139
 
                VolumeInfo volumeInfo = new VolumeInfo();
140
 
                volumeInfo.setStatus(StorageProperties.Status.creating.toString());
141
 
                List<VolumeInfo> volumeInfos = db.query(volumeInfo);
142
 
                for(VolumeInfo volInfo : volumeInfos) {
143
 
                        String volumeId = volInfo.getVolumeId();
144
 
                        LOG.info("Cleaning failed volume " + volumeId);
145
 
                        blockManager.cleanVolume(volumeId);
146
 
                        try {
147
 
                                volumeStorageManager.deleteObject("", volumeId);
148
 
                        } catch(Exception ex) {
149
 
                                LOG.warn(ex);
150
 
                        }
151
 
                        db.delete(volInfo);
152
 
                }
153
 
                db.commit();
154
 
        }
155
 
 
156
 
        public void cleanFailedVolumes() {
157
 
                EntityWrapper<VolumeInfo> db = StorageController.getEntityWrapper();
158
 
                VolumeInfo volumeInfo = new VolumeInfo();
159
 
                volumeInfo.setStatus(StorageProperties.Status.failed.toString());
160
 
                List<VolumeInfo> volumeInfos = db.query(volumeInfo);
161
 
                for(VolumeInfo volInfo : volumeInfos) {
162
 
                        String volumeId = volInfo.getVolumeId();
163
 
                        LOG.info("Cleaning failed volume " + volumeId);
164
 
                        blockManager.cleanVolume(volumeId);
165
 
                        try {
166
 
                                volumeStorageManager.deleteObject("", volumeId);
167
 
                        } catch(Exception ex) {
168
 
                                LOG.warn(ex);
169
 
                        }
170
 
                        db.delete(volInfo);
171
 
                }
172
 
                db.commit();
173
 
        }
174
 
 
175
 
        public void cleanFailedVolume(String volumeId) {
176
 
                EntityWrapper<VolumeInfo> db = StorageController.getEntityWrapper();
177
 
                VolumeInfo volumeInfo = new VolumeInfo(volumeId);
178
 
                List<VolumeInfo> volumeInfos = db.query(volumeInfo);
179
 
                if(volumeInfos.size() > 0) {
180
 
                        VolumeInfo volInfo = volumeInfos.get(0);
181
 
                        LOG.info("Cleaning failed volume " + volumeId);
182
 
                        blockManager.cleanVolume(volumeId);
183
 
                        try {
184
 
                                volumeStorageManager.deleteObject("", volumeId);
185
 
                        } catch(Exception ex) {
186
 
                                LOG.warn(ex);
187
 
                        }
188
 
                        db.delete(volInfo);
189
 
                }
190
 
                db.commit();
191
 
        }
192
 
 
193
 
        public void cleanStuckSnapshots() {
194
 
                EntityWrapper<SnapshotInfo> db = StorageController.getEntityWrapper();
195
 
                SnapshotInfo snapshotInfo = new SnapshotInfo();
196
 
                snapshotInfo.setStatus(StorageProperties.Status.creating.toString());
197
 
                List<SnapshotInfo> snapshotInfos = db.query(snapshotInfo);
198
 
                for(SnapshotInfo snapInfo : snapshotInfos) {
199
 
                        String snapshotId = snapInfo.getSnapshotId();
200
 
                        LOG.info("Cleaning failed snapshot " + snapshotId);
201
 
                        blockManager.cleanSnapshot(snapshotId);
202
 
                        try {
203
 
                                snapshotStorageManager.deleteObject("", snapshotId);
204
 
                        } catch(Exception ex) {
205
 
                                LOG.warn(ex);
206
 
                        }
207
 
                        db.delete(snapInfo);
208
 
                }
209
 
                db.commit();
210
 
        }
211
 
 
212
 
        public void cleanFailedSnapshots() {
213
 
                EntityWrapper<SnapshotInfo> db = StorageController.getEntityWrapper();
214
 
                SnapshotInfo snapshotInfo = new SnapshotInfo();
215
 
                snapshotInfo.setStatus(StorageProperties.Status.failed.toString());
216
 
                List<SnapshotInfo> snapshotInfos = db.query(snapshotInfo);
217
 
                for(SnapshotInfo snapInfo : snapshotInfos) {
218
 
                        String snapshotId = snapInfo.getSnapshotId();
219
 
                        LOG.info("Cleaning failed snapshot " + snapshotId);
220
 
                        blockManager.cleanSnapshot(snapshotId);
221
 
                        try {
222
 
                                snapshotStorageManager.deleteObject("", snapshotId);
223
 
                        } catch(Exception ex) {
224
 
                                LOG.warn(ex);
225
 
                        }
226
 
                        db.delete(snapInfo);
227
 
                }
228
 
                db.commit();
229
 
        }
230
 
 
231
 
        public void cleanFailedSnapshot(String snapshotId) {
232
 
                EntityWrapper<SnapshotInfo> db = StorageController.getEntityWrapper();
233
 
                SnapshotInfo snapshotInfo = new SnapshotInfo(snapshotId);
234
 
                List<SnapshotInfo> snapshotInfos = db.query(snapshotInfo);
235
 
                if(snapshotInfos.size() > 0) {
236
 
                        SnapshotInfo snapInfo = snapshotInfos.get(0);
237
 
                        LOG.info("Cleaning failed snapshot " + snapshotId);
238
 
                        blockManager.cleanSnapshot(snapshotId);
239
 
                        try {
240
 
                                snapshotStorageManager.deleteObject("", snapshotId);
241
 
                        } catch(Exception ex) {
242
 
                                LOG.warn(ex);
243
 
                        }
244
 
                        db.delete(snapInfo);
245
 
                }
246
 
                db.commit();
247
 
        }
248
 
 
249
 
        public void transferPendingSnapshots() throws EucalyptusCloudException {
250
 
                if(!transferredPending) {
251
 
                        SnapshotTransfer transferrer = new SnapshotTransfer(this);
252
 
                        transferrer.start();
253
 
                        transferredPending = true;
254
 
                }
255
 
        }
256
 
 
257
 
        public static void checkWalrusConnection() {
258
 
                HttpClient httpClient = new HttpClient();
259
 
                GetMethod getMethod = null;
260
 
                try {
261
 
                        java.net.URI addrUri = new URL(StorageProperties.WALRUS_URL).toURI();
262
 
                        String addrPath = addrUri.getPath();
263
 
                        String addr = StorageProperties.WALRUS_URL.replaceAll(addrPath, "");
264
 
                        getMethod = new GetMethod(addr);
265
 
 
266
 
                        httpClient.executeMethod(getMethod);
267
 
                        StorageProperties.enableSnapshots = true;
268
 
                } catch(Exception ex) {
269
 
                        LOG.error("Could not connect to Walrus. Snapshot functionality disabled. Please check the Walrus url.");
270
 
                        StorageProperties.enableSnapshots = false;
271
 
                } finally {
272
 
                        if(getMethod != null)
273
 
                                getMethod.releaseConnection();
274
 
                }
275
 
        }
276
 
        
277
 
        private class SnapshotTransfer extends Thread {
278
 
                private BlockStorageChecker checker;
279
 
                public SnapshotTransfer(final BlockStorageChecker checker) {
280
 
                        this.checker = checker;
281
 
                }
282
 
                
283
 
                public void run() {
284
 
                        EntityWrapper<SnapshotInfo> db = StorageController.getEntityWrapper();
285
 
                        SnapshotInfo snapshotInfo = new SnapshotInfo();
286
 
                        snapshotInfo.setShouldTransfer(true);
287
 
                        List<SnapshotInfo> snapshotInfos = db.query(snapshotInfo);
288
 
                        if(snapshotInfos.size() > 0) {
289
 
                                SnapshotInfo snapInfo = snapshotInfos.get(0);
290
 
                                String snapshotId = snapInfo.getSnapshotId();
291
 
                                db.commit();    
292
 
                                List<String> returnValues;
293
 
                                try {
294
 
                                        returnValues = blockManager.prepareForTransfer(snapshotId);
295
 
                                } catch (EucalyptusCloudException e) {
296
 
                                        LOG.error(e);
297
 
                                        return;
298
 
                                }
299
 
                                String snapshotFileName = returnValues.get(0);
300
 
                                File snapshotFile = new File(snapshotFileName);
301
 
                                Map<String, String> httpParamaters = new HashMap<String, String>();
302
 
                                HttpWriter httpWriter;
303
 
                                SnapshotProgressCallback callback = new SnapshotProgressCallback(snapshotId, snapshotFile.length(), StorageProperties.TRANSFER_CHUNK_SIZE);
304
 
                                httpWriter = new HttpWriter("PUT", snapshotFile, callback, "snapset-" + UUID.randomUUID(), snapshotId, "StoreSnapshot", null, httpParamaters);
305
 
                                try {
306
 
                                        httpWriter.run();
307
 
                                } catch(Exception ex) {
308
 
                                        db.rollback();
309
 
                                        LOG.error(ex, ex);
310
 
                                        checker.cleanFailedSnapshot(snapshotId);
311
 
                                }
312
 
                        }
313
 
                }
314
 
        }
315
 
}