Compendium modifications made by M.Begeman - Fall 2007 / Spring 2008page 1
Introduction
This document contains the record of code modifications made to Compendium 1.5.2 by M. Begeman for CogNexus between the Fall of 2007 and the Spring of 2008.
Color coding of the text is as follows –
- Red text: this is code Michelle is sure she added to 2.0 (because she checked the source code for 2.0).
- Pink text: Michelle is pretty sure this code has been added, but not 100%
- Black text: Michelle is not sure whether the code was added or not. In many cases she was not sure what to look for.
- Blue text: commends added by Michelle
Code Changes Completed
ICoreConstants.java - added ALPHA xx to the version string
UIStartup.java - added the name of the default database being opened to the startup dialog.
*******************************************************************
UISearchDialog.java - Bug fix where Before/After boxes were switched for "Date Modified" search
******************************************************************
DBBackupDatabase.java - Bug fix: Changed algorithm to write the backup file out a line at a time instead
of caching the whole thing in memory (failed if the database was large). Only made change to the
SQL backup code (zip and XML still need to be looked at).
DBRestoreDatabase.java - Bug fix: Changed algorithm to restore the backup file a line at a time instead
of caching the whole thing in memory (failed if the database was large). Only made change to the
SQL backup code (zip and XML still need to be looked at).
Bug fix: Added deleteAllData() method to purge the tables before doing a restore. As it was, restoring
into an existing project did not remove/replace existing data, even though the dialogs warn that
all existing data will be lost. Now, the tables are emptied before the load begins.
*******************************************************************
Performance
DBNode.java -and-
DBNodeUserState.java - Made a set of changes to modify the way unread nodes are represented in the
database. Formerly, the code created an 'unread' entry for each node, and for each user in the
NodeUserState table. This created a lot of unnecessary records and impacted performance on large
databases. The basic change is disabling this behavior, and making Compendium consider the
lack of an entry in the NodeUserState table to indicate an UNREAD status.
*** N.B. this needs to be reviewed and validated for completeness ***
*******************************************************************
INodeService.java -and-
NodeService.java -and-
DBNode.java -and-
DBNodeUserState.java - Implemented methods that will return the total number of records in the
Node table, and the total number of records for the current user in the NodeUserState table.
These are used in UIMenuView to calculate the size of the Unread View and give the user a
chance to bail if they are asking to build a huge view.
New methods are lGetNodeCount() and lGetStateCount().
Implemented parallel set of lGetViewCount() methods to issue a warning if they choose
View ==> "Map/List Views..." menu item on a large database.
UIMenuView - Changes to AddUnreadView to check the size of the task - letting the user bail!
- Changes to ActionPerformed to check size of View ==> Map/List Views request
*******************************************************************
New feature
UIUtilities.java - Modified CopyDnDfile() to implement new feature that allows an administrator
to create a system-wide override on where "Copied Linked Files" are stored. This is
useful in a true groupware setting where you want Linked Filed stored on a mapped network
drive.
Modified unzipXMLZipFile() to correctly place Zipped reference nodes in the folder
specified by the override.
XMLImport.java - Make changes to put imported reference nodes into the customized
"Copied Linked Files" folder, if any. Modifications all reference
UIUtilities.sGetPathPrefix().
This code has changed again with the adding of storing in the database as well as folders and all the LinkedFiles datamodel classes to manage it. But I believe I did add the original changes above before that
*******************************************************************
Performance
UINodeEditPanel - Replaced the usage of jFileChooser with java.awt.FileDialog(). Although
jFileChooser is a more modern interface to the file system, there are huge performance
problems with it dating back to 2004. Lots of discussion about this is on the Sun Java
discussion boards - but no good workaround. This causes very slow performance in opening
the 'contents' and 'properties' windows for Maps and Reference nodes, because they have
a 'file chooser' for icons, background, and/or the reference node target. Depending on
different things, opening one of these could take anywhere from 10 seconds to several
minutes (on Windows, at least).
I appear to be still using JFileChooser. I think there where some benefits I was not willing to give up. Maybe in newer versions of Java the original issue has improved?
*******************************************************************
UINodeEditPanel.onExecute() - Bug fix. In v1.5.2, if you double click an INR in your inbox
it will open fine, but if you open the INR's Contents (right click) and then press the
Launch button, it will fail. This has to do with onExecute() not being able to figure out
the 'History Bar' path of how we got to where we are. Fixed with a kludge that forces the
History Bar to "?" when this happens (hey, it's better than failing completely!).
I cannot tell from the code if I added this as I think it changed a lot, but the Launch button works, so I think I did.
*******************************************************************
Feature modification
UIList.java-and-
ListTableModel.java - (1) Added AUTHOR to list views; (2) Changed the default ('Less') list
view to include Creation date and Author info. This makes the Inbox easier to deal with
(i.e., it shows the creation date and author of the message)
*******************************************************************
*** Changes to speed up label typing (defer DB update until node loses focus) ***
These are important in a networked environment because 1.5.2 executes a database
update for every keystroke!
NodeSummary.java -
1) Made setLabelLocal a public method, added a bLabelDirty flag which tells us that the
label needs to be flushed to the database
2) Added flushLabel() method to flush the label to the DB if it's dirty
UINode.java -
1) Added a call to NodeSummary.flushLabel() from lostFocus() method to push the label to the
DB if necessary
2) Created a shadow setText() method that takes a boolean flag bDefer, which calls
NodeSummary.setLabelLocal() instead of NodeSummary.setLabel()
NodeUI.java - Changed the following methods to call UINode.setText() with the boolean bDefer flag:
addCharToLabel() - paste() - cut() - delete() - keypressed()(while in 'editing' state)
*******************************************************************
*** Changes that speed up opening maps, opening nodes, creating nodes & links (and maybe other stuff)
NodeSummary.java - changed the way that getNumOfMultipleViews() determines the number of
parents a node has. Now does this by calling the new method iGetParentCount()
INodeService.java -and-
NodeService.java -and-
DBNode.java Implemented new method iGetParentCount() to directly give us the number
of parents a given node has.
*******************************************************************
*** General performance mods ***
DBConnection.java - Changed MYSQL_SESSION_TIMEOUT from 28800 to 28800000. The timeout as
documented was supposed to be an 8 hour timeout. But the methods used return time in
milliseconds (28800 is 8 hours of *seconds*). This means the database connection was
being timed out every 28.8 seconds(!!!) causing a new DB connection to be initialized.
Follow-up note: This value has been scaled back to 120000 (2 minutes) because of a
server-side timeout at Pair. This causes 6 DB calls every time there's been a 2-minute
idle by the user. When this goes into deployment at SCE, this value should be scaled
up to 'match' any of the server timeouts they set. (N.B. On Jeff's ISP's MySQL server,
this number can be scaled up to 10 minutes [600000] safely.)
ProjectCompendium.App.setTrashBinIcon() - This method sets the TrashBin Icon, and changes if there are
deleted nodes in the database. This *was* accomplishing this by calling NodeService.GetDeletedNodeSummary()
which fetches all the deleted node data from the DB and builds a vector of node objects
(in turn,causing more database calls), only to count the elements in the vector and then
garbage-collect all of it. setTrashBinIcon() now calls a new method iGetDeletedNodeCount()
which gives us the count directly and saves a LOT of database activity for node/map/link deletion.
DBNode.ProcessNode - Changed three calls (GetImage,GetReference, GetImageSize) - each
resulting in a database call - to getIRIS, which does all three in one shot. This
optimization benefits opening maps, eliminating lots of database calls.
DBReferenceNode.getIRIS() - New method to implement the change to ProcessNode()
NodeService.getReaders() - Changed code from using DBUser.getUserProfileFromID() which incurs seven (7!)
database calls to a new method DBUser.getUserNameFromID() which gets the needed info (reader's name)
with one query. This speeds up opening an object's contents or properties or detail pane.
Also fixed a bug here - the code was displaying the Author field from the User table when
it should have been displaying the Name field.
*******************************************************************
Collection of updates that eliminate hitting the database for user names when
populating the node details window and the Readers pop-up
NodeService.getReaderIDs()- and -
iNodeService.getReaderIDs()New methods which return a list of an objects's reader's IDs
UserProfile.java - added getUserID() method
UINodePropertiesPanel.UpdateReadersInformation() -and-
UIReadersDialog.UpdateListView (hey, I didn't name it this...) - changed their algorithms to
use the new getReaderIDs() method and look up the name from the in-memory UserProfile list
instead of going back to the database for each reader just to get their name.
*******************************************************************
New feature
ViewPaneUI.java - Added F12 and Shift-F12 event handlers to mark an entire selection seen/unseen. [Map views]
Added methods onMarkSelectionSeen() and onMarkSelectionUnseen(), which are called by keyPressed()
UIViewPane.java - Added markSelectionSeen() and markSelectionUnseen to actually implement this.
ListUI.java - Added F12 and Shift-F12 event handlers to mark an entire selection seen/unseen. [List views]
Added methods onMarkSelectionSeen() and onMarkSelectionUnseen(), which are called by keyPressed(),
and which actually implement the functionality.
Bug fix: ListUI.openNode() - added some code that marks a Reference node (INR) as 'Seen' if you double-click
on it. In 1.5.2, the reference would launch, but leave the node marked as unseen in the list. This
was mostly problematic in your Inbox.
*******************************************************************
New feature
UINodePopupMenu.java - Added code to onCreateInternalLinkInView() that sends an internet email message
whenever someone sends a node to someone's internal Inbox.
Seems to be there but slightly different. The above function is in another dialog called UISendMailDialog. Maybe a later update, but the functionality is added.
*******************************************************************
New feature
**** Changes to implement File ==> Mark Entire Project as Seen ****
messages.properties - Added menu item #234 "Mark Entire Project as Seen"
UIMenuFile.java - Added #234 to the File menu (miMarkProjectSeen)
ProjectCompendiumFrame.java - added onMarkProjectSeen() to handle the menu item when selected, and
isProjectOpen2() helper method.
UIMarkProjectSeenDialog.java - New code for the dialog box
MarkProjectSeen.java (in com.compendium.ui for lack of a better place) - Drives the actual work
and also creates/destroys the progress bar.
INodeService.vMarkProjectSeen ==>
NodeService.vMarkProjectSeen ==>
DBNode.vMarkProjectSeen==>
DBNodeUserState.vMarkProjectSeen() - Actually implements the DB updates to mark all as seen.
*******************************************************************
FormatProperties.java - Bug fix. In v1.5.2 Compendium would pay attention to the 'displayUnreadView'
property upon startup, and display (or not) the Unread View based on whether or not it was
visible when C. closed last. However, 1.5.2 did *not* set the state of the "View => Unread View"
menu to correspond, so you'd end up with the Unread View displayed and no way to turn it off (the
only menu option was to turn it on, which gives you *another* Unread View tab.
Although one way to fix this would be to make the menu item 'checked' if the property is set, I
decided to simply force the Unread View off when C. starts. Part of the rationale for this is that
the Unread View forces a traversal of the entire reachable database starting at your home window,
which is very slow when on a network. If you're in a big database, this could mean waiting many
minutes before your home window opens.
I cannot tell if I added this as I am not sure I totally understand it.
The UIMenuView is definitely checking to see what the 'displayUnreadView' setting is and setting the menu option accordingly. So I think I have added it?
*******************************************************************
UISearchDialog.java - Removed 'Case sensitive' from the search keyword dialog, because
keyword searches are *not* case sensitive!
*******************************************************************
Feature change, bug fix & major groupware performance improvement
The "timed refresh" code, which is a groupware feature, reloads your maps periodically so you'll see
changes other people have made. As implemented in v1.5.2, what it does is throw away all data it has
for all open views, fetches the data for all open views from the database again, and rebuilds the views.
This is VERY slow (practically unusable) if your database is on the internet, and most of the time
completely unnecessary.
The following modules were modified to make Compendium keep track of the last time each view was modified
by someone else. When the timer goes off, Compendium now checks (with 1 database call) to see if
anyone has modified the view since we last loaded it. If nobody has, we're done. If a view *has*
been changed by someone, the user is given the option of reloading the project now automatically, or
manually (with the refresh button) when it is convenient.
Note: Future way to make this even better: (1) figure out how to add just the incremental changes to a view.
Data side changes...
datamodel.nodesummary.java - Added an instance variable and a getter method getLastModificationDate()
to keep track of the last modificationdate for the node.
I seem to have called it getLastModifiedDate
datamodel.view.java - Added an instance variable LastModifiedbyOther to track the last time this view
was modified by someone else. Added code to initializemembers() to update this var as a view's
nodes are read in from the database. Added bIsViewDirty() method to return whether or not the database
contains updates not reflected in the View object.
datamodel.services.IViewService - added bIsViewDirty(), which links to...
datamodel.services.ViewService - added bIsViewDirty(), which links to...
db.DBViewNode.java - added bIsViewDirty() which queries the database for updated nodes.
db.management.DBConnection.java - Changed BUSY_TIMEOUT from 2 seconds to 2 minutes. Turns out that
the bIsViewDirty() polling code is getting a shared DB connection. If the polling code (which is fast) runs
while a view is loading (a long operation), the DBConnection code was closing the Connection when the
polling code returned the connection to it, which screwed up the view loading (getting 'socket closed' errs).
Lengthening the timeout is a crude work-around, but it works.
Timer/UI changes...
ui.UIRefreshManager.java - Used to call reloadProjectData every time the timer went off. Now calls
checkProjectDirty() instead.
ui.ProjectCompendiumFrame.java - Added checkProjectDirty() code and a couple of semaphores/flags
that keep the dirty-checking code and the project-reloading code from colliding with each other.
ui.toolbars.UIToolBarData.java - changes to keep the reload button active even when the timer is
running (makes sense, now that the timer isn't causing a reload). Also added a couple of methods
so that UIToolBarManager could turn the Reload button on and off.
UIToolBarManager.java - new methods for turning the Reload button on and off. The checkProjectDirty()
code uses this to disable the Refresh button while it is checking the database for updates. This
prevents an ugly little race condition.
UIOptionsDialog.java - and -
FormatProperties.java - A pair of matched changes that implement an option (User Options / Misc panel)
so that the user can choose between the old AutoReload behavior and the new Prompt to Reload behavior.
People on fast (LAN) networks can thus benefit from AutoReload, while users using databases hostedover the public internet can choose to reload the project when they want to. Note that the new
algorithm for checking to see if a reload is necessary still applies to both - so even users who
choose the AutoReload option will be reloading ONLY when there's something new - an improvement over the way it was done in 1.5.2.
I could not see these options in the User Options / Misc panel for 2.0. Not sure why I would have not added all this?
Subsequent changes that cause ONLY the 'dirty' views to be refreshed. 1.5.2 and the above reload ALL views
that the user has opened. This refinement only rebuilds opened views that are out of date.
datamodel.nodesummary.java - added removeNodeSummaryListItem() method that allows us to remove a specific nodesummary item from the NodeSummaryList
datamodel.link.java - added removeLinkSummaryListItem() method that allows us to remove a specific
Link item from the LinkSummaryList
datamodel.view.java - added reloadViewData() method that clean out all existing view data and reloads
everything 'fresh' from the database
ui.ProjectCompendiumFrame.java - Added reloadViewData() method to drive the reloading and refreshing
of a specific View. Changed checkProjectDirty() to call reloadViewData() instead of the
heavy-weight reloadProjectData().
*******************************************************************
Feature implementation (Inactive User state) and major performance improvement. This set of changes does
several things...
1) Creates an "Inactive" user state. Why? When someone leaves the team it may seem logical to delete them
as a Compendium user. Doing this also results in losing them from the Readers list on nodes, and you cannot
search on nodes created by them. By marking them as Inactive, they remain on the Readers list, and you can still