Recovery of OLVM/oVirt Virtual Disks and Snapshots and iSCSI LUN Data

What should you do if an OLVM or oVirt virtual machine stops booting, its disks disappear from storage, or important files are accidentally deleted inside the guest operating system? In this article, we will show how to use Hetman RAID Recovery to recover deleted copies of virtual machine disks from an encrypted LVM RAID 6 volume. We will connect these disks in their current state and in the state saved in a snapshot created in OLVM. We will compare their contents and save the required files without starting the virtual machine.

Recovery of OLVM/oVirt Virtual Disks and Snapshots and iSCSI LUN Data

The OLVM (Oracle Linux Virtualization Manager) and oVirt virtualization platforms use a complex storage infrastructure that includes virtual disks, snapshots, storage domains, and iSCSI network storage. Server failure, file system corruption, errors when working with storage, or failure of physical drives can result in loss of access to virtual machines and critical data.

In this article, we will examine how to recover OLVM/oVirt VM virtual disks, snapshots, and iSCSI LUNs after storage failures or corruption. We will analyze the structure of storage domains, the relationships between base images and snapshots, and methods for locating and restoring virtual disks. Special attention will be paid to working with iSCSI LUNs, connecting damaged storage, and recovering data without the risk of overwriting the original information.

Go to view
How to Recover OLVM/oVirt Virtual Machines, Snapshots & iSCSI LUNs

How to Recover OLVM/oVirt Virtual Machines, Snapshots & iSCSI LUNs

STEP 1. Preparing the test environment

For the demonstration, we prepared a test environment on a Dell PowerEdge R710 server running Oracle Linux 9.8.

Component Specifications and purpose
Server Dell PowerEdge R710
Form factor 2U rack server
Processors Up to two Intel Xeon 5500 or 5600 series processors
CPU architecture Intel 64, x86-64
RAM Up to 288 GB DDR3 ECC Registered (depending on configuration)
Memory slots 18 DIMM
RAID controller Dell PERC 6/i or H700 (depending on configuration)
Supported RAID levels RAID 0, 1, 5, 6, 10, 50, 60 (depending on controller)
Drives SAS or SATA HDD/SSD, depending on the disk subsystem configuration
Network interfaces Built-in Gigabit Ethernet ports
Expansion slots PCI Express
Operating system Oracle Linux 9.8
Linux kernel UEK or RHCK, depending on the selected kernel
File systems XFS, ext4, and other file systems supported by Oracle Linux
Virtualization KVM/QEMU
Server management Dell iDRAC 6
Purpose Server virtualization, data storage, working with virtual machines and network storage

Five physical drives are connected to the server. Using LVM, three RAID 6-type logical volumes were created on them. Each of them is distributed across all five drives, encrypted separately using LUKS2, and formatted with the ext4 file system.

Three RAID 6 logical volumes were created using LVM

The second volume contains the OLVM NFS storage with three Windows 10 virtual machine disks: a 20 GB system disk and two additional 2 GB disks (thin and thick).

OLVM NFS storage with three virtual machine disks

After populating the disks with test files, we created a virtual machine snapshot that captured the state of all three disks. Later, we deleted some files from both additional 2 GB disks inside the Windows guest. This will allow us to compare the current contents of these disks with the state saved in the snapshot.

Separately, to verify recovery, we copied the NFS storage folder together with the disk files, snapshot, and related metadata to the third encrypted volume, and then deleted this copy.

The NFS storage folder together with the disk files, snapshot, and related metadata

Two Direct LUNs from Synology were also connected to Windows 10, with capacities of 1 GB and 2 GB. In the guest system, they appeared as separate drives, from which we also deleted some files.

Their data was not included in the virtual machine snapshot, so these LUNs will be analyzed separately.

STEP 2. Connecting the disks

Proceed to the computer with Hetman RAID Recovery installed.

For recovering data from failed RAID arrays, NAS devices, and other multi-disk storage systems.

First, we will recover copies of the OLVM virtual machine disks. For this purpose, we will use sector-by-sector images of the server’s five physical drives. They contain a complete sector-level copy of the drives, including areas where deleted data may remain, and allow analysis without repeated access to the original disks.

Hetman RAID Recovery: sector-by-sector images of the server's five physical drives

If the drives are healthy, they can be connected directly to the computer through the appropriate controller or a compatible adapter. However, if the drives are damaged or unstable, it is better to create full images first and perform further analysis on them.

Hetman RAID Recovery: mounted images of the server's five physical drives

STEP 3. Decrypting LUKS2 and analyzing the file system

After the disks are connected, Hetman RAID Recovery analyzes the LVM metadata and automatically detects three RAID 6 logical volumes. They appear on the main screen in the RAID Volumes section.

Hetman RAID Recovery analyzes LVM metadata and automatically detected RAID 6 logical volumes

We can see the LUKS label next to the volumes, which means their contents are encrypted. We select the third volume, from which the copy of the OLVM storage folder was deleted, and open it by double-clicking. In the Unlock Disk window, we enter the password or specify the key file to unlock the volume.

Hetman RAID Recovery: We can see the LUKS label next to the volumes, which means their contents are encrypted

After unlocking, the program detects the ext4 file system and opens the analysis type selection wizard. We leave Fast scan selected, click Next, and wait for the scan to complete.

Hetman RAID Recovery: Fast scan

In our case, this mode is sufficient to find deleted folders. If the required data is not found in the results, you can repeat the analysis by selecting Full analysis and the Ext2/3/4 file system. This search is more thorough and takes more time.

When the process is complete, click Finish and proceed to the found directory structure.

Hetman RAID Recovery: found directory structure

STEP 4. Searching for and recovering OLVM disks

In the analysis results, open the found vm-storage folder. Inside it, go to the storage domain folder with the UUID name, and then to images.

Hetman RAID Recovery: found directory structure

Here, we can see folders and files with long UUID identifiers. This is how OLVM stores virtual disks and their states. Alongside the disk files are related metadata files that help determine their parameters and relationships.

Hetman RAID Recovery: folders and files with long UUID identifiers

To connect the disks together with the snapshot later, save the entire vm-storage folder, including its subfolders and metadata. This will allow the program to find related files and display the available states of each disk.

Return to the parent folder, select vm-storage, and click Recovery. In the wizard, choose saving to a hard drive and specify a destination folder on another physical drive.

Hetman RAID Recovery: select vm-storage and click Recovery

Do not save recovered files to the drives from which recovery is being performed: this may overwrite data that still needs to be restored.

Start the save operation and wait for it to complete. Next, we will connect the recovered storage folder to the program to analyze the virtual machine disks and their snapshot.

STEP 5. Connecting the recovered disks as images

The storage folder has been saved. Now we will connect the virtual machine disks together with the snapshot to access the files inside them.

Click Mount Disk. In the Virtual machine disk images section, select QEMU and click Next.

Hetman RAID Recovery: In the Virtual machine disk images section, select QEMU

In the selection window, specify the entire recovered vm-storage folder, which in our environment was used as OLVM NFS storage, and click Select Folder. The program checks its contents, finds the virtual disk files, and determines the relationships between their states.

Hetman RAID Recovery: In the selection window, specify the entire recovered vm-storage folder

In the next window, we can see three groups:

  • two additional 2 GB disks;
  • thin and thick;
  • a 20 GB system disk.

For each one, the snapshot state is shown, marked as Snapshot #1, and the current state is marked as Active. For comparison, we select both states of the two additional disks and click Next.

Hetman RAID Recovery: Snapshot #1

After they are mounted, they appear on the main screen as four separate drives. The program detects NTFS partitions on them with the labels THIN-2GB and THICK-2GB. Each label appears twice because we mounted two states of the corresponding disk.

Now each of these states can be opened and analyzed like a regular disk to view and save the required files.

Hetman RAID Recovery: NTFS partitions with the labels THIN-2GB and THICK-2GB

STEP 6. Analyzing the disks and their snapshots

We will start with the current state of one of the additional disks. Open its NTFS partition by double-clicking it. In the analysis wizard, select Fast scan, click Next, and after completion click Finish.

Hetman RAID Recovery: In the analysis wizard, select Fast scan

Go to the General folder with the test data. Here you can see both existing files and files deleted while working in Windows. The program marks deleted items with a red cross.

Hetman RAID Recovery: General folder with test data

Now open the snapshot state of the same disk and also run Fast scan. After completion, go to the same folder.

In the saved state, the files that were later deleted are displayed without red crosses. At the time the snapshot was created, they were still present on the disk, so their earlier versions can be recovered here.

The analysis of the current state and the snapshot of the second additional disk is performed in the same way.

Check the contents of the required files in the preview window. The program allows you to preview documents, images, archive contents, and video files.

Select the required files from the current state or from the snapshot, click Recovery, and save them to another physical drive.

Hetman RAID Recovery: select the required files and click Recovery

STEP 7. Connecting Synology and searching for LUN files

We have already analyzed the OLVM disks and their snapshot. Now we will move on to the two Direct LUNs from the Synology storage, from which some files were also deleted in Windows.

We connect Synology to Hetman RAID Recovery via SSH.

Hetman RAID Recovery: Connect Synology to Hetman RAID Recovery via SSH

After the connection is established, the program displays the physical disks and volumes available on the NAS. In the list, select the main Synology volume, mark it, and click Next. The program checks its structure and adds the volume to the drive list.

Hetman RAID Recovery: the program displays the physical disks and volumes available on the NAS

Open it by double-clicking and start a quick scan.

Hetman RAID Recovery: the program displays the physical disks and volumes available on the NAS

On our Synology, iSCSI LUN data is stored in the @iSCSI directory. In the nested LUN directory, there are FILE and THIN folders: the first LUN is located in FILE, and the second is located in THIN.

Hetman RAID Recovery: In the nested LUN directory, there are FILE and THIN folders

Open the directory with the UUID name in each folder. In this test setup, each LUN is represented by a single file that contains the block data of the disk previously connected to Windows 10. Select both files, click Recovery, and save them to another physical drive.

Hetman RAID Recovery: Open the directory with the UUID name in each folder

STEP 8. Analyzing LUN images and recovering deleted files

After saving LUN 1 and LUN 2, connect them to Hetman RAID Recovery as disk images. Click Mount Disk, select RAW disk images, and proceed to the next step.

Hetman RAID Recovery: Click Mount Disk, select RAW disk images

Because the LUN files do not have a standard image file extension, change the filter to All files in the selection window and add both files one by one.

Hetman RAID Recovery: change the filter to All files in the selection window

The mounted images appear in the drive list. The program detects their NTFS partitions.

Hetman RAID Recovery: mounted images appear in the drive list

Open the first LUN partition, run a quick scan, and then repeat these steps for the second one. In the results, you can see both existing files and data deleted while working in Windows 10. The program marks such items with a red cross.

Hetman RAID Recovery: existing files and data deleted while working in Windows 10

Find the required documents, photos, archives, and video recordings, and verify their contents in the preview window.

Select the files you want to recover, click Recovery, and save them to a physical drive.

Conclusion

In this article, using Hetman RAID Recovery, we recovered the deleted copy of the OLVM storage from an encrypted LVM RAID 6 volume. The program automatically detected the volume structure, and after entering the password, provided access to its contents.

We connected the recovered disks together with the virtual machine snapshot and compared their states. In the current state, we found the deleted files, and in the snapshot, the same files were available before deletion. The required data was viewed and saved without starting Windows or rolling back the virtual machine.

Separately, we connected Synology via SSH, saved the files of two Direct LUNs, and analyzed them as disk images. This made it possible to recover the data deleted from them in the guest system.

Vladimir Artiukh

Author: , Technical Writer

Vladimir Artiukh is a technical writer for Hetman Software, as well as the voice and face of their English-speaking YouTube channel, Hetman Software: Data Recovery for Windows. He handles tutorials, how-tos, and detailed reviews on how the company’s tools work with all kinds of data storage devices.

Oleg Afonin

Editor: , Technical Writer

Oleg Afonin is an expert in mobile forensics, data recovery and computer systems. He often attends large data security conferences, and writes several blogs for such resources as xaker.ru, Elcomsoft and Habr. In addition to his online activities, Oleg’s articles are also published in professional magazines. Also, Oleg Afonin is the co-author of a well-known book, Mobile Forensics - Advanced Investigative Strategies.

Recommended For You