CODEWITHSUNDEEP
amazon-s3amazon-ec2amazon-web-services
Lauren Dahlin
Lauren Dahlin

Reputation: 159

Snapshot vs. Volume Size

I am using a public dataset snapshot in Amazon ec2. The data in the snapshot is roughly 150GB and the snapshot itself is 180GB. I knew that by performing operations on the dataset I would need more than 30GB of free memory so I put the snapshot in a 300GB volume. When I look at my stats though (unfortunately as a process is running, so I think I am about to run out of room), it appears that the snapshot is still limited to 180 GB.

  1. Is there a way to expand its size to the size of the volume without losing my work?

  2. Is there a possibility that the snapshot is actually continuous with another drive (e.g. /dev/sdb)? (A girl can hope, right?)

    Filesystem Size Used Avail Use% Mounted on
    /dev/sda1 9.9G 1.1G 8.4G 11% /
    none 34G 120K 34G 1% /dev
    none 35G 0 35G 0% /dev/shm
    none 35G 56K 35G 1% /var/run
    none 35G 0 35G 0% /var/lock
    none 35G 0 35G 0% /lib/init/rw
    /dev/sdb 827G 201M 785G 1% /mnt
    /dev/sdf 174G 162G 2.6G 99% /var/lib/couchdb/0.10.0

My instance is running Ubuntu 10.

Upvotes: 1

Views: 1887

Answers (1)

Steffen Opel
Steffen Opel

Reputation: 64741

Is there a way to expand its size to the size of the volume without losing my work?

That depends on whether you can live with a few minutes downtime for the computation, i.e. whether stopping the instance (hence the computation process) is a problem or not - Eric Hammond has written a detailed article about Resizing the Root Disk on a Running EBS Boot EC2 Instance, which addresses a different but pretty related problem:

[...] what if you have an EC2 instance already running and you need to increase the size of its root disk without running a different instance?

As long as you are ok with a little down time on the EC2 instance (few minutes), it is possible to change out the root EBS volume with a larger copy, without needing to start a new instance.

You have already done most of the steps he describes and created a new 300GB volume from the 180GB snapshot, but apparently you have missed the last required step indeed, namely resizing the file system on the volume - here are the instructions from Eric's article:

Connect to the instance with ssh (not shown) and resize the root file system to fill the new EBS volume. This step is done automatically at boot time on modern Ubuntu AMIs:

# ext3 root file system (most common)
sudo resize2fs /dev/sda1
#(OR)
sudo resize2fs /dev/xvda1

# XFS root file system (less common):
sudo apt-get update && sudo apt-get install -y xfsprogs
sudo xfs_growfs /

So the details depend on the file system in use on that volume, but there should be a respective resize command available for all but the most esoteric or outdated ones, none of which I'd expect in a regular Ubuntu 10 installation.

Good luck!

Appendix

Is there a possibility that the snapshot is actually continuous with another drive (e.g. /dev/sdb)?

Not just like that, this would require a RAID setup of sorts, which is unlikely to be available on a stock Ubuntu 10, except if somebody provided you with a respectively customized AMI. The size of /dev/sdb does actually hint towards this being your Amazon EC2 Instance Storage:

When an instance is created from an Amazon Machine Image (AMI), in most cases it comes with a preconfigured block of pre-attached disk storage. Within this document, it is referred to as an instance store; it is also known as an ephemeral store. An instance store provides temporary block-level storage for Amazon EC2 instances. The data on the instance store volumes persists only during the life of the associated Amazon EC2 instance. The amount of this storage ranges from 160GiB to up to 3.3TiB and varies by Amazon EC2 instance type. [...] [emphasis mine]

Given this storage is not persisted on instance termination (in contrast to the EBS storage we all got used to enjoy - the different behavior is detailed in Root Device Storage), it should be treated with respective care (i.e. never store something on instance storage you couldn't afford to loose).

Upvotes: 1

Related Questions