The NMS system is HA Solaris U2000 V100R003C00SPC102.
One day during the maintenace check we found the database table TNSDHDB becomes ABNORMAL = reach the full capacity = 1000MB = 100% usage.
The faulty database is possible to find in SysMonitor Wndow - Database Monitor - The database. It is in abnormal state has RED dot and the state is ABNORMAL. In this state customer can have problem to configure new SDH services.
The procedure of databse expansion is complex, so please use Guide from attachment.
R&D test the database and find that the database table tAlarmToTrail can be reconstruct...949536 KB to 5712 KB
We tried to reconstruct the database to decrease the database table size by below procedure:
# . /opt/sybase/SYBASE.sh
root@Secondary # isql -Usa -SDBSVR -P
1> use TNSDHDB
1> reorg rebuild tAlarmToTrail
Beginning REORG REBUILD of table 'tAlarmToTrail'.
Msg 1105, Level 17, State 16:
Server 'DBSVR', Line 1:
Can't allocate space for object 'tAlarmToTrail' in database 'TNSDHDB' because
'default' segment is full/has no free extents. If you ran out of space in
syslogs, dump the transaction log. Otherwise, use ALTER DATABASE to increase the
size of the segment.
So this procedure fail due to NOT enough space in database --> so the only choice is to increase the database size. (R&D had little bit earlier database data so there were able to reconstruct the database)
When the database becomes ABNORAMAL due to full capacity the solution can be reconstruction of database or expansion.
After the R&D analyzes the root cause of this problem is:
So many end2end operations of data creation and deleting caused many rags. R&D did the test, they found the size will be increased if the same data is created and deleted.
No further final solution now.