1.Objective
To provide a guide on how to recover an FCS with a load file that causes loads and tuning parameter saves to fail with the error;
ERR:ERROR= 11186:The Generation time of master data base and FCS image is different.
This indicates that the time stamp in the master copy of the FCS program, which is stored on the EWS, no longer matches the time stamp in the actual FCS. This is normally caused by an EWS crash or severe network failure during an online load.
The conventional solution to this (and the recommended Yokogawa Solution) is to wait until the FCS can be offline loaded. Where this is not possible, the FCS master file time stamp can be edited to match the FCS time stamp. This will allow tuning parameter saves and further loads.
When it is suspected that a load has failed and as a result the FCS/EWS timestamp does not match, rather than attempting another load to prove this – it is preferable to select a tuning parameter save. This will preserve the working files from the failed load. If this completes successfully then there is no problem with the load – look elsewhere than this procedure! However, if the parameter save fails, then you need to follow this procedure (note - the Tuning parameter save may not complete if there is a time stamp difference, and an engineering restart of the EWS will be required).
Note; This procedure will only work as long as there have been no tags added or deleted and are consequentially different between the EWS copy and the running FCS master.
This procedure has been used many times to successfully enable online loads to be performed after a time stamp mismatch. It will not provide a full solution when tags lists mismatch between the EWS and the FCS. In this circumstance the FCS will not allow online loads to the FCS Area or station with the mismatched tag list (the FCS will reject the load). The FCS will allow online loads to other Areas within the FCS. The good news is that an Area offline load can be completed in a minute or less and the I/O does not fail safe on an Area offline load.
Though Areas with mismatching tag lists cannot be online loaded, once the time stamp has been repaired tuning parameter saves can be carried out.
2.DISCLAIMERS and WARNINGS
The author takes no responsibility for failures of plant and control hardware as a result of users following this procedure. It is written to be as comprehensive a guide as possible – however there is no doubt that every crash/recovery has its own unique characteristics – and hence own unique pitfalls.
Users attempting this must be confident with the Yokogawa file structure, work and master files, usage of builder tools and HP unix in general.
3.RECOVERY STEPS
Depending on the circumstance of the crash use the following checklist as a recovery guide;
- Backup the full project to tape using builder tools
- Have there been any further edits to master files since the load failed.
YES = You must identify each file that was edited and return it to the state it was in when the load failed, ie; New work files must be deleted, work files that have been further modified – you need to return the file to the exact same state it was in when the load failed (restore from backup, re-edit the master etc).
- Backup the Failed FCS using unix, ie;
cd /home/builder/PROJECT/real
cp –Rp FCS.ii FCS_Bak01.ii
- Are there any locked files in the crashed FCS?
Use /opt/ydcs/eng/bin/fmlockl "u:builder p:PROJECT r" or locklist to check if files are locked.
If there are locked files either unlock them using /opt/ydcs/eng/bin/fmulock -l –k [file key] or;
unlock [Host Name]. Failing this carry out an engineering shutdown and restart.
4. TIME STAMP UPDATE
This procedure describes the tools and techniques required to edit the FCS master binary image (stored on the EWS) such that it’s timestamp matches the timestamp in the FCS. As the edited data is six words, four words for a time stamp and the remaining two words are a checksum.
- Locate the EWS utility dbdump, ie; find / -name dbdump –print – should return /opt/ydcs/eng/bin/hp9k700/dbdump (several versions will exist in different paths – this version works for HPUX 10.20)
#su [root password]
#cd /opt/ydcs/eng/bin/hp9k700
#ls –al dbdump
-r-xr-xr-x 1 root root 518184 May 8 1996 dbdump
#./dbdump –h E0321 16 3(Gateway to use, Station of FCS, Domain of FCS)
Data base start logical address = 0x01000000 –
(Takes about five – ten minutes to access the FCS database)
03-16-main:area=all>mds #StnCns
1: SEBOLƒXƒe [ƒVƒ‡ƒ“’è ”ƒe [ƒuƒ‹ utclStnCns 00000000 000200( 512)
2: ƒXƒe [ƒVƒ‡ƒ“‹¤’Ê’è ”ƒe [ƒuƒ‹ StnCnstTbl 01000100 000280( 640)
Select NO ?(1-2) =2 (Select the area which corresponds to StnCnstTbl)
ƒXƒe [ƒVƒ‡ƒ“‹¤’Ê’è ”ƒe [ƒuƒ‹ StnCnstTbl 01000100 000280( 640)
01000100 5354 4E20 434E 5354 36CB 6DB2 C4F7 0000 STN CNST6.m.....
01000110 0000 0000 0000 0000 0000 07D0 003C 003C ...... <.<
01000120 0000 0000 0000 0000 0310 0040 0001 0000 ...... @....
01000130 0000 0000 0000 0000 0000 0000 0000 0000 ......
01000140 0001 0005 0004 0001 0258 0000 0000 0000 ...... X......
01000150 0000 0000 0000 0000 0000 0000 0000 0000 ......
01000160 0000 0000 0000 0000 0000 0000 0022 0400 ...... "..
01000170 0020 012C 0008 0000 0000 0000 0000 0000 . .,......
01000180 0000 4000 0100 0000 0000 03E8 1388 0002 ..@......
01000190 003C 0000 0000 0000 0000 0000 0000 0000 .<......
010001A0 42C8 0000 0000 000A 01F4 01F4 1000 03E8 B......
010001B0 003C 0008 000A 000A 0032 0000 0000 0000 .<...... 2......
010001C0 0001 9000 0100 0200 0000 0000 0000 0000 ......
010001D0 0000 0000 0000 0000 0000 0000 0000 0000 ......
010001E0 0040 0007 0001 000A 0000 0000 0000 0000 .@......
010001F0 0000 0000 0000 0000 0000 0000 0000 0000 ......
03-16-main:area=all>mds #A1-areaCns
ƒGƒŠƒA’è ”ƒe [ƒuƒ‹ A1-areaCnstTbl 01100200 000100( 256)
01100200 4143 4E53 5420 3031 4853 3131 0000 0000 ACNST 01FCS1....
01100210 36CD 5200 0955 0000 0000 0000 0000 0000 6.R..U......
01100220 0000 0000 0000 0000 0000 0000 0000 0000 ......
01100230 0000 0000 0000 0000 0000 0000 0000 0000 ......
01100240 0DAC 068B 07D0 066A 0067 0016 03E8 0100 ...... j.g......
01100250 0000 0000 000A 000A 0014 0007 0064 0000 ...... d..
01100260 0006 0100 0000 0000 0000 0032 0064 0000 ...... 2.d..
01100270 0000 0000 0000 0000 0000 0000 0000 0000 ......
01100280 00C8 01F4 3F00 0000 3F40 0000 3F60 0000 ....?...?@..?`..
01100290 42C8 0000 4100 0000 0000 0000 0000 0258 B...A...... X
011002A0 0002 0001 003C 0000 0000 0000 0000 0000 .....<......
011002B0 0000 0000 0000 0000 0000 0000 0000 0000 ......
011002C0 0000 0000 0000 0000 000A 000A 000A 000A ......
011002D0 0000 0000 0000 0000 0000 0000 0000 0000 ......
011002E0 0000 0000 0000 0000 0000 0000 0000 0000 ......
011002F0 0000 0000 0000 0000 0000 0000 0000 0000 ......
03-16-main:area=all>quit
The sections in BOLD contain the unix time stamps / checksums in the header – hang on to these listings they’ll be need in the next sections.
- Station configuration files – Editing the EWS Image
The examples below are for a healthy load file – this demonstrates the relationship between the FCS load and the EWS image. In a file that has an incorrect timestamp, the bolded sections will have differing values.
The hex editor is packaged with the Yokogawa tools – any hex editor could be used.
#cd /home/builder/PROJECT/real/FCS.ii/load/master/load
#ls
AREADEF.l IOMDEF.l SFDATA.l TFDATA.l
#/opt/ydcs/eng/bin/hp9k700/lmm - SFDATA.l
DOS68K Load Module Modify Version 0.3
> mds 300
00000300 5354 4E20 434E 5354 36CB 6DB2 C4F7 0000 STN CNST6.m.....
00000310 0000 0000 0000 0000 0000 07D0 003C 003C ...... <.<
00000320 0000 0000 0000 0000 0310 0040 0001 0000 ...... @....
00000330 0000 0000 0000 0000 0000 0000 0000 0000 ......
00000340 0001 0005 0004 0001 0258 0000 0000 0000 ...... X......
00000350 0000 0000 0000 0000 0000 0000 0000 0000 ......
00000360 0000 0000 0000 0000 0000 0000 0022 0400 ...... "..
00000370 0020 012C 0008 0000 0000 0000 0000 0000 . .,......
00000380 0000 4000 0100 0000 0000 03E8 1388 0002 ..@......
00000390 003C 0000 0000 0000 0000 0000 0000 0000 .<......
000003A0 42C8 0000 0000 000A 01F4 01F4 1000 03E8 B......
000003B0 003C 0008 000A 000A 0032 0000 0000 0000 .<...... 2......
000003C0 0001 9000 0100 0200 0000 0000 0000 0000 ......
000003D0 0000 0000 0000 0000 0000 0000 0000 0000 ......
000003E0 0040 0007 0001 000A 0000 0000 0000 0000 .@......
000003F0 0000 0000 0000 0000 0000 0000 0000 0000 ......
> m 308
Identify the corrsponding section in the #StnCns unpload – enter the hex data for the timestamp and checksum
00000308 36CB '6.' ? 36CB
0000030A 6DB2 'm.' ? 6DB2
0000030C C4F7 '..' ? C4F7
0000030E 0000 '..' ? .
> mds 300
00000300 5354 4E20 434E 5354 36CB 6DB2 C4F7 0000 STN CNST6.m.....
00000310 0000 0000 0000 0000 0000 07D0 003C 003C ...... <.<
00000320 0000 0000 0000 0000 0310 0040 0001 0000 ...... @....
> q
Check the entered data to confirm the undate has been successful.
Note 36CB6DB2 is hex for the number of seconds since 1/1/70 GMT – and refers to the last compile time of this file. This is easy to check in MsAccess, use the command line in debug;
? DateAdd("S", Cdbl("&H36CB6DB2") + 8*3600,#1/1/70#)
18/02/1999 9:32:34
- Area configuration files – Editing the EWS Image
#cd /home/builder/PROJECT/real/FCS.ii/FCS1.ii/load/master/load
#ls
OGDATA.l PRDATA.l SFDATA.l TFDATA.l
#/opt/ydcs/eng/bin/hp9k700/lmm - SFDATA.l
DOS68K Load Module Modify Version 0.3
> mds 400
00000400 4143 4E53 5420 3031 4853 3131 0000 0000 ACNST 01FCS1....
00000410 36CD 5200 0955 0000 0000 0000 0000 0000 6.R..U......
00000420 0000 0000 0000 0000 0000 0000 0000 0000 ......
00000430 0000 0000 0000 0000 0000 0000 0000 0000 ......
00000440 0DAC 068B 07D0 066A 0067 0016 03E8 0100 ...... j.g......
00000450 0000 0000 000A 000A 0014 0007 0064 0000 ...... d..
00000460 0006 0100 0000 0000 0000 0032 0064 0000 ...... 2.d..
00000470 0000 0000 0000 0000 0000 0000 0000 0000 ......
00000480 00C8 01F4 3F00 0000 3F40 0000 3F60 0000 ....?...?@..?`..
00000490 42C8 0000 4100 0000 0000 0000 0000 0258 B...A...... X
000004A0 0002 0001 003C 0000 0000 0000 0000 0000 .....<......
000004B0 0000 0000 0000 0000 0000 0000 0000 0000 ......
000004C0 0000 0000 0000 0000 000A 000A 000A 000A ......
000004D0 0000 0000 0000 0000 0000 0000 0000 0000 ......
000004E0 0000 0000 0000 0000 0000 0000 0000 0000 ......
000004F0 0000 0000 0000 0000 0000 0000 0000 0000 ......
> m 0410
00000410 36CD '6.' ? 36CD
00000412 5200 'R.' ? 5200
00000414 0955 '.U' ? 0955
00000416 0000 '..' ? .
> mds 400
00000400 4143 4E53 5420 3031 4853 3131 0000 0000 ACNST 01FCS1....
00000410 36CD 5200 0955 0000 0000 0000 0000 0000 6.R..U......
00000420 0000 0000 0000 0000 0000 0000 0000 0000 ......
00000430 0000 0000 0000 0000 0000 0000 0000 0000 ......
> q
Check the entered data to confirm the undate has been successful, note 36CD 5200 is hex for the number of seconds since 1/1/70 GMT – and refers to the last compile time of this file.
One or the other, or both, of the two calculated dates above will correspond to the date on the ICS system page for the FCS.
This procedure must be carried out for each Area in the FCS Station.
- Recovery from Errors
If it is suspected that errors are introduced during the edit procedure, restore the file from your local copy and try again.
5. ONLINE LOAD
The FCS should now accept an online load. Prove this by selecting a tuning parameter save.
Before carrying out the online load please re-read section 2 – DISCLAIMERS AND WARNINGS.
6. ICS UPDATES
The ICS Master File update will not have completed with the modifications. This update is incremental and is normally conducted at the end of an FCS load. To ensure that the ICS are kept up to date;
- Carry out a full ICS Master load for each ICS
OR
- Undo/modify slightly each function block, or tag source (I/O, Switches, Alarms) – that was changed in the previous failed load. This will force the next load to do an ICS master file update for these files.
OR
- Carry out item 1. For a single ICS and use this as a template for other ICSs by copying files at a unix level – experienced users only !!!
C:\Sites\Automated\Local\downloads\FCS_Time_Stamp.docPage 1 of 4