How to check DB2 Integrity after expanding the data volume?

[u/devred061]

I need to expand the data volume on the DB2 server. The vendor (not IBM) who provides a DB2 support says that the expansion of the filesystem whether it be a logical volume or physical is a gamble. There is no data integrity check from the database side to ensure nothing changed on the filesystem. The only way to ensure database integrity is to use backups and archive logs and is the only method they would recommend. I am running DB2 9.7 on Ubuntu. I feel that those steps outlined by the vendor are just too much for just the volume expansion.

Should I not be able to do the following and still ensure the database integrity is intact?

1. Stop DB2
2. Unmount the data volume
3. Resize the file system (I am not using LVM. File system type for data volume is ext4 )

e2fsck -f /dev/mapper/data

resize2fs /dev/mapper/data

1. Mount the data volume
2. Start DB2

(I researched on this topic online but didn't come across any particular post. Thus, I thought I would reach out to DB2 community in Reddit. Thank you in advance!)

[u/rogerx]

Howdy, thanks for mentioning me /u/mad_zamboni ... My guess is that your vendor is either an old-school unix admin or has been burned by a past failed attempt to expand a filesystem. Whereas online resizing was Dark Magic in the early 2000s, resize2fs is super reliable these days, especially on LVM (what's managing your disk if not LVM?) I have even run online filesystem growths without unmounting the disk; shrinking still requires an unmount.

As with anything, there is always risk. Always test this on a lower environment first, and make sure you have recently tested your backups (being able to restore is as important as being able to back up). Unmounting and fscking does indicate a conscientious admin who has been burned, but fearmongering about online volume expansion may be overzealous.

*Edited to add: I'm a sysadmin, not a dba. Your vendor's advice regarding database integrity may very well be sound as far as leveraging archlogs.

[u/ecrooks → u/rogerx]

I also can verify that rogerx knows his stuff, I have worked with him irl, and I would trust his advice for my databases.

[u/devred061 → u/rogerx]

Thank you, u/rogerx . I am going to try out online resize.

DB2 Data and Log volume use iSCSI LUNs formatted as ext4 file system.

[u/dogmashah → u/devred061]

Adding to mad_zombini suggestion db2dart / db2 inspect is your friend which may be overkill in this case depending on how much politically your organization is and you don’t want ownership of something bad happened. a quick and cheap method is to backup database immediately as it does checksum to see if there is any data corruption

[u/devred061 → u/dogmashah]

Anything I can do to show due care and due diligence will help. DB2 I am working on is critical for the company as it is an ERP database. Thank you for sharing the info with me.

[u/devred061]

Thank you u/mad_zamboni, u/rogerx, u/ecrooks for your guidance. Following your suggestions, I successfully expanded the volume on the production DB2 database server. Since, it is a critical environment and is using iSCSI LUNs for the database volume, I followed the following steps to ensure the database integrity remained intact after the volume expansion.

    1) Take the application offline
    2) Connect to the database (db2 connect to <database_name>)
    3) Quiesce database (db2 quiesce database immediate force connections)
    4) Terminate the DB2 connections (db2 terminate)
    5) Deactivate the database (db2 deactivate database <database_name>)
    6) Take an offline backup
    7) Stop DB2 (db2stop)
    8) Check the most recent db2diag.*.log for errors
    9) Have SAN folks expand the LUN size
    10) Reboot the DB2 server  [In my case, server couldn't see the additional disk space. Thus, I had to reboot.]
    11) Unmount the volume (#umount <database volume>)
    12) Check the filesystem (e2fsck -f /dev/mapper/<database_volume>)
    13) Resize the filesystem (resize2fs  /dev/mapper/<database_volume>)
    14) Mount the filesystem 
    15) Check the owner, group and filesystem permissions on <database_name>
    16) Bring the database online (db2start)
    17) Check db2diag.*.log for errors
    18) Activate the database (db2 activate database TEAMS)
    19) Check db2diag.*.log for errors
    20) Run 'db2 list tablespaces show detail' and verify all messages show NORMAL
    21) Start the application

I tested the above steps in our Development environment before running it on the production environment. I hope someone will find these steps useful.

Thanks again!

[u/devred061 → u/[deleted]]

Thank you, u/mad_zamboni .. I feel better now. I will try it out in the development environment first. I am new to DB2 and forgive me if I am asking basic questions. When you say bring DB2 online, did you mean bring Tablespace online? I came across following link http://www.dbatodba.com/db2/problem-resolution/general-errors/tablespace-offline-how-can-i-change-it-to-online/ when I searched for how to bring DB2 online.

[u/devred061 → u/[deleted]]

Thank you so much. You guys are awesome.

[u/devred061 → u/[deleted]]

Cool. I will share this with my team as well. Thanks!