BSD DevCenter

oreilly.comSafari Books Online.Conferences.

We've expanded our LAMP news coverage and improved our search! Search for all things LAMP across O'Reilly!

Search
Search Tips

advertisement

Listen Print Discuss Subscribe to BSD Subscribe to Newsletters

Using Software RAID-1 with FreeBSD
Pages: 1, 2

Booting into the Mirror/Duplex

If you watch your boot-up messages, you should see this in bold white text right after the disks are probed:



GEOM_MIRROR: Device gm0 created (id=2125638583).
GEOM_MIRROR: Device gm0: provider ad0 detected.
GEOM_MIRROR: Device gm0: provider ad0 activated.
GEOM_MIRROR: Device gm0: provider mirror/gm0 launched.
GEOM_MIRROR: Device gm0 already configured.
Mounting root from ufs:/dev/mirror/gm0s1a

and the system will continue to boot. However, if you have a typo in /etc/fstab, the boot will stop at this point and wait for you to type something meaningful. In this example, I forgot to insert mirror when I edited /etc/fstab, meaning /dev/gm0s1a should have been /dev/mirror/gm0s1a so that FreeBSD could find my root filesystem:

Mounting root from ufs:/dev/gm0s1a
setrootbyname failed
ffs_mountroot: can't find rootvp
Root mount failed: 6

Manual root filesystem specification:
  <fstype>:<device>  Mount <device> using filesystem <fstype>
            e.g. ufs:da0s1a
  ?             List valid disk boot devices
  <empty line>       Abort manual input

mountroot>

Fortunately, that's not as scary as it looks. Start by listing your valid disk boot devices:

mountroot> ?

List of GEOM managed disk devices:
  mirror/gm0s1f mirror/gm0s1e mirror/gm0s1d mirror/gm0s1c mirror/gm0s1b 
mirror/gm0s1a mirror/gm0s1 ad2s1 mirror/gm0 ad0s1 ad2 acd0 ad0 fd0

If you type in the correct location of the / filesystem, the system will continue to reboot:

mountroot> ufs:/dev/mirror/gm0s1a
Mounting root from /dev/mirror/gm0s1a

After logging in, be sure to edit the offending line in /etc/fstab and try rebooting again. When you can boot up and log in successfully, verify that each partition on the mirror mounted successfully with:

% df -h
Filesystem            Size    Used    Avail    Capacity    Mounted on
/dev/mirror/gm0s1a    248M     35M     193M       15%        /
devfs                 1.0K    1.0K       0B      100%        /dev
/dev/mirror/gm0s1e    248M     12K     228M        0%        /tmp
/dev/mirror/gm0s1f    7.3G     99M     6.7G        1%        /usr
/dev/mirror/gm0s1d    248M    196K     228M        0%        /var

df won't show your swap partition; you can verify it with:

% swapinfo
Device                1K-blocks    Used    Avail    Capacity
/dev/mirror/gm0s1b       629544       0   629544        0%

Synchronizing the Mirror/Duplex

The only thing left to do is to synchronize the data on both hard drives. This will happen automatically as soon as you issue the command to insert the second drive into the mirror:

# gmirror insert gm0 /dev/ad2
GEOM_MIRROR: Device gm0: provider ad2 detected.
GEOM_MIRROR: Device gm0: rebuilding provider ad2.

To see what's happening:

# gmirror list | more
Geom name: gm0
State: DEGRADED
Components: 2
Balance: round-robin
Slice: 4096
Flags: NONE
GenID: 0
SyncID: 1
ID: 2125638583
Providers:
1. Name: mirror/gm0
   Mediasize: 10262568448 (9.6G)
   Sectorsize: 512
   Mode: r6w5e2
Consumers:
1. Name: ad0
   Mediasize: 10262568448 (9.6G)
   Sectorsize: 512
   Mode: r1w1e1
   State: ACTIVE
   Priority: 0
   Flags: DIRTY
   GenID: 0
   SyncID: 1
   ID: 3986018406
2. Name: ad2
   Mediasize: 10262568448 (9.6G)
   Sectorsize: 512
   Mode: r1w1e1
   State: SYNCHRONIZING
   Priority: 0
   Flags: DIRTY, SYNCHRONIZING
   GenID: 0
   SyncID: 1
   Synchronized: 1%
   ID: 1946262342

Note the SYNCHRONIZING on the Flags line. It will take a while for these two drives to synchronize, as it is currently at 1 percent. I've seen times ranging from about 30 minutes for a 10GB drive to about two and a half hours for a 75GB drive. If you're curious, check the progress with:

# gmirror status
Name    Status    Components
mirror/gm0    DEGRADED    ad0
            ad2 (2%)

You'll see a status message in bold white text when the synchronization finishes:

GEOM_MIRROR: Device gm0: rebuilding provider ad2 finished.
GEOM_MIRROR: Device gm0: provider ad2 activated.

If you repeat gmirror list, you'll note that the State has changed from DEGRADED to COMPLETE and the Synchronized line is now gone. Don't worry if you see DIRTY on the Flags line, as it simply indicates that the system has written new data to the disk but hasn't mirrored it yet. If you were to wait a few seconds on a quiet disk, you would see the Flags line change to NONE.

For the final test, reboot the system.

This time your startup messages should include:

GEOM_MIRROR: Device gm0 created (id=2125638583).
GEOM_MIRROR: Device gm0: provider ad0 detected.
GEOM_MIRROR: Device gm0: provider ad2 detected.
GEOM_MIRROR: Device gm0: provider ad0 activated.
GEOM_MIRROR: Device gm0: provider ad2 activated.
GEOM_MIRROR: Device gm0: provider mirror/gm0 launched.
Mounting root from ufs:/dev/mirror/gm0s1a

Final Notes

GEOM utilities are works in progress, and the developers constantly add new features and updates to the man pages. It's well worth your while to keep your favorite version of FreeBSD up-to-date using cvsup or to choose a newer release when deciding which version of FreeBSD to install.

If you wish to gather performance statistics on your mirror/duplex, try gstat(8). A good read through gmirror(8) is also in order, especially if you want an overview of the procedure for replacing a failed disk.

Dru Lavigne is a network and systems administrator, IT instructor, author and international speaker. She has over a decade of experience administering and teaching Netware, Microsoft, Cisco, Checkpoint, SCO, Solaris, Linux, and BSD systems. A prolific author, she pens the popular FreeBSD Basics column for O'Reilly and is author of BSD Hacks and The Best of FreeBSD Basics.


Read more FreeBSD Basics columns.

Return to theBSD DevCenter


Have a question about your setup? Posting here may get you answers, or at least interesting questions.
You must be logged in to the O'Reilly Network to post a talkback.
Post Comment
Full Threads Oldest First

Showing messages 1 through 23 of 23.

  • Provider ad2 too small.
    2008-08-06 21:15:42  linuse [Reply | View]

    Hi,
    I am trying to do RAID1 mirror on live freebsd6.0 system. Both harddisk are 40GB of size. When I type gmirror insert gm0 /dev/ad2 it gaves me "Provider ad2 too small" message. What's wrong? Earlier ad2 hardisk was running Winxp and freebsd dual OS. I completly deleted the partition and tried to do RAID1 but that errror "Provider ad2 too small" message comes up.
    Sysinstall command shows both hardisk size of 38162MB. Output of the fdisk command on both hardisk:

    computer1# fdisk /dev/ad0
    ******* Working on device /dev/ad0 *******
    parameters extracted from in-core disklabel are:
    cylinders=77545 heads=16 sectors/track=63 (1008 blks/cyl)

    Figures below won't work with BIOS for partitions not in cyl 1
    parameters to be used for BIOS calculations are:
    cylinders=77545 heads=16 sectors/track=63 (1008 blks/cyl)

    Media sector size is 512
    Warning: BIOS sector numbering starts with sector 1
    Information from DOS bootblock is:
    The data for partition 1 is:
    sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD)
    start 63, size 78156162 (38162 Meg), flag 80 (active)
    beg: cyl 0/ head 1/ sector 1;
    end: cyl 1023/ head 254/ sector 63
    The data for partition 2 is:
    <UNUSED>
    The data for partition 3 is:
    <UNUSED>
    The data for partition 4 is:
    <UNUSED>


    bhw-proxy.hons.com.np# fdisk -s /dev/ad2
    fdisk: invalid fdisk partition table found
    fdisk: read_s0: Unknown error: 0
    computer1# fdisk /dev/ad2
    ******* Working on device /dev/ad2 *******
    parameters extracted from in-core disklabel are:
    cylinders=77542 heads=16 sectors/track=63 (1008 blks/cyl)

    Figures below won't work with BIOS for partitions not in cyl 1
    parameters to be used for BIOS calculations are:
    cylinders=77542 heads=16 sectors/track=63 (1008 blks/cyl)

    fdisk: invalid fdisk partition table found
    Media sector size is 512
    Warning: BIOS sector numbering starts with sector 1
    Information from DOS bootblock is:
    The data for partition 1 is:
    sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD)
    start 63, size 78162273 (38165 Meg), flag 80 (active)
    beg: cyl 0/ head 1/ sector 1;
    end: cyl 741/ head 15/ sector 63
    The data for partition 2 is:
    <UNUSED>
    The data for partition 3 is:
    <UNUSED>
    The data for partition 4 is:
    <UNUSED>

    Any suggestions? Please
  • Great Article
    2008-07-16 13:26:44  pu239 [Reply | View]

    Thanks for the great article, easy to use and setup.
  • Remaining Disk Space wrong
    2007-12-21 14:29:27  wsoeldner [Reply | View]

    I'm new to RAID. I have set up a FreeNAS Software RAID 1 NAS in an old computer. I have two 200GB Seagate Drives installed on the array. I access it through two windows Vista Laptops. freeNAS set it up fine, and it was showing 186 (or so) GB of space. No issues there.

    I was consolidating all my files off of other drives to the array when I ran out of disk space. It's possible, as I was just trying to get everything in one place to start sorting through it. Disk space used 180GB, free space 700Kb.

    Since then I've moved about 70GB off of the array and I'm still only showing 706Kb free space. When I select all files/folders on the share and right-click properties, Total size of data is 111GB. When I right click the actual share it shows it with no space.

    I imagine it's a sync issue, but not knowing much about FreeBSD, I'm clueless on how to fix the problem.

    Thanks and Happy Holidays

    Warren
  • URL with a mirror status script.
    2007-08-10 18:47:25  DrkShdw [Reply | View]

    I was looking at gmirror, and was doing a basic script to check for a failed array. then I stumbled across a nice customizable script that someone had written. It works really well for multiple arrays in a single machine. (It'd need slight modification for more than 1 array)

    Find it here: http://www.monkeybrains.net/~rudy/example/raid_status.html
  • disklabel
    2007-05-24 23:01:41  ttschn [Reply | View]

    hi,
    thanks for the guide - worked like a charm, now i have geom raid1 from install!
    just one thing -- is anyone else having problems with the gm0s1 disklabel? i'm getting errors complaining about partition c not starting at 0 and not filling the whole disk, and partition g (the last partition) exceeding the end of the disk
  • fstab error
    2006-07-11 18:21:37  alvinph [Reply | View]

    Hello everyone.

    Thank guys. I was able to implement the software RAID-1 in our FreeBSD box. The system boots up okey and I was able to login without any problems. But during the boot process I noticed these lines

    Jun 28 15:52:32 afrim fsck: fstab: /etc/fstab:3: Inappropriate file type or format
    Jun 28 15:52:32 afrim fsck: fstab: /etc/fstab:5: Inappropriate file type or format
    Jun 28 15:52:32 afrim fsck: fstab: /etc/fstab:7: Inappropriate file type or format
    Jun 28 15:52:32 afrim fsck: fstab: /etc/fstab:9: Inappropriate file type or format
    Jun 28 15:52:32 afrim fsck: fstab: /etc/fstab:11: Inappropriate file type or format


    Other than that the system work fine. Can anyboy shed my some light on my these lines apper on the boot-up process?

    Thanks
    • fstab error solved
      2006-07-18 23:53:15  alvinph [Reply | View]

      Hi guys i finally figured out what was causing the problem. I went back to my fstab configuration and found the problem. Just had to move some lines and aligned some configurations.
    • Dru Lavigne photo fstab error
      2006-07-12 11:06:13  Dru Lavigne | O'Reilly Author [Reply | View]

      It's hard to tell without seeing a copy of your /etc/fstab. Also, what version of FreeBSD?
      • fstab error
        2007-03-18 19:05:41  fraggle.rok [Reply | View]

        I had the same initial problem with the "inappropriate filetype or format". I am installing FreeBSD 6.2

        Examining /etc/fstab revealed that my edits had caused the last 2 fields (Dump, Pass#) to wrap onto a new line! Carefully editing /etc/fstab to put the values back on the same line corrected the problem - verified that the messages did not appear after a reboot.
  • mountroot> stuck here can't go any further
    2006-04-01 16:48:13  bsdnoob [Reply | View]

    After following all the instructions and rebooting I'm stuck at the mountroot> prompt. ? shows this;

    ad2s1f ad2s1e ad2s1d ad2s1c ad2s1b ad2s1a ad0s1f ad0s1e ad0s1d ad0s1c ad0s1b ad0s1a ad2s1 ad0s1 acd0 ad2 ad0 fd0

    There's no mirror/gm stuff. Now I can't get into my system. Now what to I do to recover my system or get this working? I've tried entering these in but no luck.
  • Use of >> is safer than >
    2006-03-22 10:58:49  n4hhe [Reply | View]

    It is, however, useful to tell the operating system to load it whenever you boot. This requires edits to two files. The first one is currently empty, so just echo over the required line:

    # echo geom_mirror_load="YES" > /boot/loader.conf


    I would suggest it best to be in the habit of using '>>' rather than '>' except in situations one expressly desires a new file to be started from scratch. Its too likely something else is already in /boot/loader.conf to use '>'.
  • What if one of the drives fail?
    2006-01-28 21:15:30  ayam666 [Reply | View]

    The article is good, simple to follow, but does not actually explain the procedure of replacing the hard disk if any one of them fails.

    Someone has posted a similar question and it seems that no one (including the author) can answer it yet.

    Googling it returns nothing either. Maybe because it is a new thing and not many people have implemented it yet.

    What a shame.
    • Dru Lavigne photo What if one of the drives fail?
      2006-01-29 08:21:19  Dru Lavigne | O'Reilly Author [Reply | View]

      Actually "man gmirror" has some easy to follow examples if a drive fails. While I haven't had a drive fail, in my testing I physically removed a drive and the rebooted OS kept perking along. I then replaced the drive with an empty one and was successfully able to restore the mirror. One of those things that is better to try once on a test system rather than wait until an emergency :-)
  • error when load ad2
    2006-01-13 00:51:48  logza [Reply | View]

    Plz help me,
    how to fix
    proxy# atacontrol list
    ATA channel 0:
    Master: ad0 <SAMSUNG SP0411N/TW100-13> ATA/ATAPI revision 7
    Slave: no device present
    ATA channel 1:
    Master: ad2 <SAMSUNG SP0411N/TW100-13> ATA/ATAPI revision 7
    Slave: no device present

    proxy# gmirror insert gm0 /dev/ad2
    Cannot access provider ad2.

    Thanks
    • Dru Lavigne photo error when load ad2
      2006-01-18 07:25:28  Dru Lavigne | O'Reilly Author [Reply | View]

      Are you repairing a broken mirror? If so, don't forget to run "gmirror forget" before running the "gmirror insert" command.

      If this is a new mirror, you need to run "gmirror label" first. Take a close look at the EXAMPLES section of "man gmirror".
  • something missing?
    2005-12-28 14:59:44  corbesero [Reply | View]


    I was very pleased to come across this article because it looked like a very direct way to set up raid on the system disks.

    I just tried this on a new machine using a FreeBSD 6.0 release cd, but it failed. :( Specifically, it gave me the "operation not permitted" message whenI tried to execute the "gmirror label" command. I did set the kern.debugflags.geom to 16, so I felt somewhat stumped.

    Is there anything else that needs to be done?

    • something missing?
      2006-03-30 12:20:47  ragnar_spain [Reply | View]

      it also happened to me, but i found the solution, before running the gmirror label run this:
      # gmirror load

      and also is suggesting to do the next comand to the hds:
      # gmirror clear adX

      another tip is that if u already installed freebsdm u can also add gmirror through the installation disk entering the option to fix the system and then:
      # chroot /dist
      # mount_devfs devfs /dev
      ... gmirror comands ...
      # mount /dev/mirror/gm0s1a /mnt
      noticing that ur system will be at /mnt
  • What if Drive ad0 fail?
    2005-12-11 15:12:50  aboyz [Reply | View]

    Hi,

    Was wondering what if one drive failed does the gmirror still work and will it alert you to replace?

    For example. if ad0 failed. does ad2 take over until you replace ad0? or visa versa? Kinda like a RAID CARD where one drive fail you can rebuild it thru the RADI CARD interface.

    How does the gmirror replace the drive? can someone explain the process to replace a bad HD in the future?

    Thanks
  • small change required if drives not ad0 & ad2
    2005-12-08 20:38:34  molivier [Reply | View]

    Great, simple article on gmirror, thank you! Super helpful compared to other stuff out there.

    One improvement I'd suggest - where I stumbled a little - I have SATA drives ad4 and ad6, so where you say:

    "Change each ad to a gm, and insert a mirror after /dev. For example, /dev/ad4s1a becomes /dev/mirror/gm0s1a. Unless you've made extra partitions, you'll have ad4s1 devices ending in a, b, d, e, and f and will need to edit each of those lines."

    ...I needed to change e.g., ad4 to gm0 ... I just replaced ad with gm and rebooted and got stuck for a bit in "mountroot>" purgatory :)
    • small change required if drives not ad0 & ad2
      2006-09-12 06:21:59  gein [Reply | View]

      I was about to post about this when I read your reply. The guide should read "Change each adX to a gm0".
  • avoid typos
    2005-12-07 13:53:33  scrib [Reply | View]

    Great article!

    One improvement, to back up and edit fstab in one command:

    # sed -i .orig 's/ad0/mirror\/gm0/' /etc/fstab

    (This edits fstab in-place after creating a backup as fstab.orig. Confirmed in 6.0-RELEASE, should work in others.)
  • RAID card suggestion?
    2005-11-14 08:02:34  fak3r.com [Reply | View]

    Dru - another great article, thanks so much for your time. After a superblock error on /var a few weeks back, I've decided to rebuild FreeBSD 6.0 with RAID 1 for redundant/mirroring. What RAID card would you recommend for this -or- where could I find a list of cards? I've looked on freebsd.org under hardware for 6.0, but I'm wondering which is more recommended.

    Thanks again

    faker
    • RAID card suggestion?
      2005-11-29 11:55:51  Carnavour [Reply | View]

      We use LSI MegaRaid 320-2x which is SCSI based in our FreeBSD 5.4 Server. LSI even has company support for FreeBSD. The card is the best.

      I installed the Card and connected the SCSI backplane to it which connects the hotswap drives. I then entered the LSI utiliy to create an array. After this was done. I installed FreeBSD 5.4, there were no problems, nothing special to do, just installed it normally. It just appeard as one disk. I loaded all the ports, configured the server, Finally I tested.

      I tested the RAID array be pulling out a hotswap drive while FreeBSD was running. I left it out, ran some commands, and changed some files.

      Everything worked perfectly. Then I placed the drive back in and it immediately started to sync back with the other drive. I then rebooted the OS just to check. No problems. No performance degradation either.

      I do recommend getting a optional battery for the RAID card. This will increase reliability during cache writes if the power fails. Of course this is not a specific problem to LSI all RAID cards that perform cache writes suffer from this.

      Hope this helps. Here is the link to LSI

      http://www.lsilogic.com/products/megaraid_scsi/index.html


Sponsored Resources

  • Inside Lightroom
Advertisement

Sponsored by:

O'Reilly Media

©2009, O'Reilly Media, Inc.
(707) 827-7000 / (800) 998-9938
All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners.
About O'Reilly
Academic Solutions
Authors
Contacts
Customer Service
Jobs
Newsletters
O'Reilly Labs
Press Room
Privacy Policy
RSS Feeds
Terms of Service
User Groups
Writing for O'Reilly
Content Archive
Business Technology
Computer Technology
Google
Microsoft
Mobile
Network
Operating System
Digital Photography
Programming
Software
Web
Web Design
More O'Reilly Sites
O'Reilly Radar
Ignite
Tools of Change for Publishing
Digital Media
Inside iPhone
O'Reilly FYI
makezine.com
craftzine.com
hackszine.com
perl.com
xml.com

Partner Sites
InsideRIA
java.net
O'Reilly Insights on Forbes.com