AWS FreeBSD instance won’t come up. Screams for manual fsck.
This is a short write up if after a reset or a reboot your FreeBSD (or Linux) instance doesn’t come online, stalls and
“aws ec2 get-console-output” returns something like that among its lines:
UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
- Just power-off the faulty instance either from Web interface or using cli:
aws ec2 stop-instances --instance your_instance_id
- Again, using Web interface or cli (if you know its id) detach volume the root filesystem lives on.
aws ec2 detach-volume --volume-id faulty_volume_id --instance your_instance_id
- Create a new minimal instance and attach the volume that was detached during the previous step.
- Boot it up and simply run fsck manually as advised.
For FreeBSD you will have to add an entry into /etc/fstab otherwise fsck would complain:# fsck -y /dev/xbd5a fsck: Could not determine filesystem type
In my case I just add a single line:
echo "/dev/xbd5a /mnt ufs rw 1 1" >> /etc/fstab
- After that, just do the reverse: detach the volume and connect it back to your main instance and power it up.
Hope everything is golden at this point.
on May 30, 2016 at 11:19 pm
· Permalink
Thanks, you saved me a panic attack
on May 30, 2016 at 11:27 pm
· Permalink
Hi Philip,
No worries.
Thanks, for letting me know it worked for you.
Cheers.
BR,
Sergey
on June 15, 2016 at 6:30 am
· Permalink
Awesome tip, thanks for the help on this and saving me from lots of panic or rebuilding from backups!!
on June 15, 2016 at 7:56 am
· Permalink
Heya, Justin.
Thanks for stopping by and leaving a comment.
I’m excited to hear this smallish tip worked for you and saved you from restoring.
Cheers.
Sergey
on November 9, 2016 at 7:30 pm
· Permalink
[…] was totally useless with its succinct single-worded output – “Output”. Last time when I had a similar issue after another upgrade I at least was able to glean some helpful […]
on April 18, 2018 at 12:14 am
· Permalink
I find it odd/annoying that after all these years, AWS still doesn’t provide a simple interactive “serial” console (even if limited to TERM=dumb). Not that a basic in webpage vt100 emulator is beyond Amazon’s abilities.
on April 18, 2018 at 6:56 pm
· Permalink
Can’t agree more. The lack of any console support and a blank response from “aws ec2 get-console-output” another day after OS upgrade triggered that “enough is enough” moment and I eventually kissed AWS goodbye.