Quick Survey Requirement met using XPages

David Clover  26 April 2013 12:11:44

We were asked to put together quickly a survey form which will be used in-house to collect details relating to 'External Events' requesting the levels of support required for them.

The survey form had a number of 'conditional' elements - some dialogue boxes were to appear only when certain checkbox options were chosen by the respondent. Some single questions had single 'radio button' choices and some single questions had a requirement for multiple check boxes to indicate preferences. The whole had to be secure and visible to a respondent only after logging in to our Campus authentication system (which we call SAMS) and it had to recognise the user to allow the form-filling to proceed.

My colleague, Jef Lay, assembled an suitable form within a day using standard XPages controls. The form will collect the data and a separate view just accessible to the requestor will show responses and allow download of the responses into an Excel format for subsequent manipulation.

There was no comparable option available at short notice and once again we've proved how efficient and versatile the Domino XPages model is when it comes to working with web-enabled environments.

Here's a snapshot of the form which is also visible at our Website to internal staff.

• Comments [2]

Web Delivered Resources for Maths sites

David Clover  17 April 2013 11:16:07

We are working with our Mathematics Programme area to develop a web-driven 'Assets' utility on which we can record all the web-based resources that we make available to students and enquirers.

We have a number of Maths-related websites which we need to keep track of. These also contain a number of web-delivered objects - PDF files, images, documents etc - that we want to 'single source' from a central point. When we update the resource we want them to be visible in the new version on all the related sites wherever they are.

My colleague Jacques has built a IBM Domino XPages system (his first!) to allow the Maths Programme administrative staff to enter and edit, via a web input form, all the details that make up the multitude of web-based information sources we have put out for public view.  Jacques and Jef have worked out a way to give each PDF (or other reference object stored in the system for public consumption) a unique 'short code' which will work in perpetuity to avoid the problem of over-long machine-generated URLs.

We'll be using a full range of XPages controls and security techniques to allow the in-house users to search for items and read and define the context for each item. We can use the same database as the source for PDF and other objects that need to be referenced on other sites. These will be uploaded as needed to an 'asset' form. We'll also have utilities to 'find' items (we'll use a 'reducing' dynamic find technique built into XPages for this) and bring forward for action items which need reviewing at end-of-life or at the change of a presentation from one version to another.

We have set the security to allow authorised people read/write access, and we've used Domino's powerful IP filtering tools to allow in-house computer users to view and search the items whilst inhibiting external users from seeing it unless they are authorised as editors. This is very simple given IBM Domino's pre-existing and powerful range of security options and features which are fully accessible through XPages.

Jacques has had great success in manipulating the XPage controls for paging, and has managed to fix things so that after viewing an item the 'Home' button returns the user to the same point in the list that was used to call up an item. But he has had to use a customised 'pagerSaveState' enhancement that he's devised which seems to work better than Domino's built-in one.

University staff can review our test read-only site.

For others, here are some screenshots of the work in progress.

• Comments [0]

A Corporate Discussion area

David Clover  7 February 2013 21:15:28

We've been reviewing an excellent IBM Domino 9 template from OpenNTF which seems to offer a well-conceived, well-programmed and generally useful 'Social' utility which we can exploit to create small online 'communities' for research and other projects.

The look and feel of the 'Intrapages' environment from Thomas Adrian in Sweden owes a lot to existing public social networking environments - a strength in that the conventions people have already learned from using them in their private time are readily applied here. The template uses IBMs Domino 9 beta server which we have installed (it can also use Domino 8.5.3 with the OpenNTF Extension Library) and it is written using IBMs immensely powerful 'Xpages' Java-based programming environment. It's a professionally-crafted and, so far robust, easy-to-understand  interface. Here's an example:



The template offers 'Self Signup' - a distinct plus as it allows people external to the University to take part in the discussion - as well as offering an integrated Domino Directory sign up for organisations with users who reply on Domino for other workflow and mail-enabled applications. However, to meet all our needs, it will also need to offer Active Directory signon using LDAP. If that can be added to the design it has potential to become a seriously useful tool for us in providing services to researchers on projects. In fact we already created an Active Directory/LDAP login on our locally modified version of Declan Lynch's 'Large File Sender' template a while ago so I hope it should be feasible to add it as an option here too.

Installing and configuring this powerful tool, a single .nsf database with no external dependencies, took all of 20 minutes from a standing start. Using Thomas Adrian's clear instructions, the parameterisation and tailoring of the interface was extremely easy to do. It offers the option of integrating logins with public networks such as Facebook and Linked in as well, though we shan't be doing that just yet.

I've been critical in the past of some of the offerings at OpenNTF which have promised much but then been incompletely finished, buggy or not maintained, but this one feels as though it shares the same build and programming qualities we found a long time ago in Steve Castledine's 'Domino Blog' which is now released by IBM with its Domino server as an official 'core' template (indeed this blog uses it).

There are lots of organisations out there with a Domino infrastructure for whom this could become a really important tool in exploiting the significant power (and backend simplicity) of IBM's Domino infrastructure. Thomas Adrian has made a test environment available at http://www.intrapages.com/intra.nsf/start.xsp where you can try it out.

• Comments [0]

Windows 8 - making it useful on a desktop machine

David Clover  7 January 2013 16:01:51

I've taken the plunge and replaced Windows 7 on my home machine with Windows 8 32-bit. It's an older AMD Athlon 64 processor 3700+ 2.21 Ghz, bought 16-Jun-2006. To make the Windows 8 system usable (and as initially installed, the Windows 8 'Modern' or 'Metro' interface simply isn't appropriate for a conventional desktop user) I've applied a few simple tweaks which have transformed the experience.

I could have installed the 64-bit version but wanted to compare what the 32-bit version did with the Windows 7 install. The installation did delete all the pre-existing programs on the machine, but kept all the personal settings and data so although it took a little while to reload everything, for the most part everything came back up properly including my MS Outlook Mailboxes and settings. It's sometimes worth starting from scratch to optimise the machine so that wasn't too much hardship.

Now that's done I have what feels like a fast, slick, and so far very robust, Windows 32-bit environment whose performance in all areas seems to surpass the previous Windows 7 build on exactly the the same hardware.

I very much like the improved approach to file backup (now called 'File History') using Windows's 'Libraries', though I had to specifically include my drive E: files disk area into a new 'Library' to force it to use the new feature which now backs up to a 1Tb external drive. The video codecs seemed improved and there's now no 'jumping' when looking at MP4 HD video files from my camera.

---

'Taming' Windows 8
1.        The first thing I did was to install the free Start W8 utility. This useful piece of software has received a Microsoft Compatibility rating and ensures that on starting, Windows 8 goes straight to the 'proper' Windows environment and not to the dreadful new 'Metro/Modern' interface. Everything you knew and loved about using Windows 7 works just like it used to. There's a wide choice of Start menu replacement utilities for Windows 8, but many seem to be over-complex for an ordinary user. However, 'Start W8' was very simple, elegant and more than adequate.


2.        Secondly I installed the 'God Mode' folder which gives easy access to all the settings and configurations in Windows 8 from a single location. You'll rarely (if ever) need to struggle with Microsoft's nonsensical 'Charms' bar after configuring this.

---

I now have a fast booting machine which is available in less than 30 seconds (Windows 8 is very fast indeed to boot). I've been using Windows for 20 years and It feels very robust and efficient. I was pleased to find that the on-board network card that used to sulk and not work every time on start-up in Windows 7, fires up properly at every boot now. The only failure was a copy of IBM's  Sametime 8.5.2 won't start - and IBM can't advise as it's not been tested on the platform yet. Fortunately I have a work around using IBM's new Notes 9 beta which performs flawlessly in Windows 8. I did however need to tweak the settings on my Cisco VPN installation before it would work.

Now I rarely, if ever, have to visit the ill-conceived 'Metro/Modern' interface' other than to convince myself now and then that I still don't need it. I can use all my old software, hardware (scanners, cameras, printers etc) just as I always did. The apps supplied for the 'Metro/Modern' interface are poorly conceived and simply don't work well even in the single screen 'Microsoft Window' environment that they are nominally designed for. Jakob Neilsen's criticisms are well expressed. as are those of Phillip Greenspun.

It's clear that Microsoft has taken a massive wrong turning in configuring Windows 8 for the desktop world. It's clear also, being fair, that the OS engineers have done an excellent job in bringing Windows 7 forward to version 8 as a reliable, improved and effective core platform. The wrong turning lies in the disastrous marketing decision (which I suspect many Microsoft insiders regret - especially the technically-savvy ones) to prioritise the absurd 'Metro/Modern' interface for the conventional desktop as well as for the tablet and phone.

Now I can safely regard the 'Metro/Modern' environment as an interesting but essentially non-intrusive 'sideshow' and get on productively with the things I need to do on a conventional multi-tasking Windows desktop just as I have been doing since the first version was released.

There's a long 'rant' by Brian Boyko on Windows 8 which is well worth a look. Everything Brian says about the 'Metro/Modern' interface is 100% true, but by carrying out the few tweaks mentioned above, I have safely bypassed all or most of the pain it represents and got myself  a much better 'conventional' Windows desktop computer than I had before! The Windows 8 Survival Guide video is also worth looking at, the same team's 'Windows 8: it's Almost Not Terrible' which is less formal (and more fun).

• Comments [0]

Locating OCR taxonomic errors in scanned vintage books

David Clover  19 December 2012 13:43:19

We are working on a very interesting and exciting JISC funded project which is concerned with the concept of 'Community Curation'.

Scanned Corpus of old books
Briefly, there is a large corpus of books and material printed in the 19th C and early 20th C which contain highly detailed and technical taxonomic descriptions of plants and animals in the natural world. Many of these books have been scanned recently by OCR. Now the original page images and a word by word breakdown of the OCR for each page is available to researchers. The problem is that like all OCR, there are mistakes in the scanning which prevent it from being used as a reliable and searchable resource for present and future researchers.

OCR Comparison - protoype tools
The ultimate aim is to submit the OCR'd version alongside the original page to experts in the bio-community. They will work through and correct the faults in the OCR online. When the process is complete, this will make the very detailed content of the material available electronically. Because the words are technical, and because the OCR errors in reading old fonts are not always predictable, once it is complete, there will be an accurate and searchable archive of material which will assist research into biodiversity.

Our task has been to make some prototype tools which would allow the original material and the OCR results sit side by side so that someone can work through and check the taxonomic terms and correct them where needed.

Techniques used
Using our IBM Notes/Domino server and rapid development tools, we have imported the images and OCR text into a database and produced some browsing tools which the Research Group can use to inspect and review the accuracy of the scanning. The programming time from concept from delivery of the prototypes was around 4 days.

Examples
(both examples are best viewed using a Webkit browser such as Safari or Google Chrome)

• Initial 'day 1' simple proof of concept
• Current working model

CSS and HTML Methods
The method is accomplished by HTML5 (CSS & Javascript, backwards-compatible to HTML4+CSS3+JS), and involves using the page image as the background graphic of a table. When a word is selected using the buttons or mouseover, Javascript resizes and restyles the cells of the table to move them to the new position and appearance. Additionally, the background may be offset from the default origin, and the table resized, in order to reduce the visible area of the page to a more manageable size.

In more CSS3-capable browsers, the transitions are all smoothly animated, and edges of a sub-page selection are smoothly faded by means of a gradient overlay. In IE, the only non-CSS3-capable browser, the transitions are instantaneous and edges are hard, but everything degrades gracefully. It works fully in any WebKit browser such as Safari or Chrome, and fully without transitions in IE9. Firefox uses a different method to set background image offsets.

Parsing XML input data
The data was supplied in the form of a readily parseable XML file representing all the scanned words and the parallel original images were supplied in a jpg graphics format. We have experimented with both LotusScript and Java methods to import the items into an IBM Notes database. The LotusScript method is faster, although the Java method is likely to have more resilience to changes in the XML schema. The LotusScript method gives startlingly quick results for large volumes. The Java code has been developed separately using IDE tools and then simply loaded into the Notes database to run there. In the database each word is contained in an IBM Notes a document containing the parameters for its location on the page and book. Because it is a readily exploitable database it will be straightforward to alter the stored reading to correspond with the actual word in the graphic after human analysis and selecting from a series of taxonomic alternatives. The CSS and web code behind the working model is dynamic and changes on the fly to give a 'live' feel.

Parallels elsewhere
Similar work is being done by the Australian 'TROVE' project and Google Books uses some similar techniques to assist in searching for text in antique scanned books.

Our Team
The programming work has been done by Jef and Henryk and testing done by Jacques and myself.

The next phase will be to work with the project's researchers who are analysing the OCR errors heuristically and devising an experimental interface for testers as part of their Community Curation Taxonomy project.

Storage and management of back-end data
Here is an image of the database after importing the XML data:


And here is a typical Notes document after parsing. the original XML source line is retained for reference, and fields are reserved for corrected words, and can be added for other analytical purposes as needed later:


IBM's Notes/Domino has a Document-Oriented data model which can be quickly exposed to the web with only minimal programming. It's one that we have been working with for a long time and it really comes into its own here. it is very similar in concept to the NoSQL 'CouchDB' model developed by Damian Katz, formerly of the IBM Lotus Division.

Notes/Domino is excellent also for working with 'loosely structured' data of this kind. The 'industrial' power and capacity available in IBM Notes/Domino could readily allow it to support many simultaneous sessions and updates as part of the community activity of general 'improvement' in the accuracy of the scanned content.

• Comments [0]

Windows 7, web edit and File locking

David Clover  26 October 2012 13:50:19

We have just solved a very bizarre problem involving saving files with certain extensions in Windows 7 on the server.

If we create a file with the extension 'txt - e.g. newtextfile.txt - we can readily edit it. If we rename it to newtextfile.htm, any attempt to open the file results in a lock being placed on it which prevents us from saving it under the same name. Notepad reports: "The process cannot access the file because it is being used by another process". Yet on closing the file it can be renamed in the Windows Explorer.

We have discovered that by switching off the 'Details' and 'Preview' option in the Windows File Explorer (under the 'Organize' command), this issue does not arise.

We assume therefore that when those two options are enabled in Windows 7, the Preview and Details functions in Windows Explorer are placing a file lock on the file preventing it from being edited where there is a file viewer for htm being called.

The problem does not arise in Windows XP.

• Comments [0]

Windows 8 experiments

David Clover  10 October 2012 12:47:31

 We have been experimenting with the released version of Windows 8 to see how it might stand up to general office use. My first thoughts are that the results are extremely disappointing. I cannot see how this could become a desktop operating system of choice in a corporate/institutional environment for a long time; it is a pretty challenging change of gear which will even baffle the average home desktop user. Gartner, has confirmed that very few corporates will use this as a default system, though it will probably have immediate niche applications. Computer interface guru Jakob Nielsen has conducted a thorough review of the usability of Windows 8 and finds its dual nature poorly conceived.

Although when first seen in a showroom it is quite 'pretty' and has some impressive technical improvements behind the scenes, for office-based people, long experienced in the use of computers, it is a very hard change of gear to contemplate. The well-publicised and most grievous casualty in Windows 8 in terms of the professional user is the complete loss of the convenient and integrated 'Start' menu which, in Windows 7 and previous versions back to Windows 95, gives access neatly and economically to all the software and features users need using an effective 'nested' visual metaphor.

No 'Start' button or menu provided

In a decision of some arrogance and which I think Microsoft will come to regret, the new Windows 8 ''Start Screen' presents a rambling, full-screen muddle containing some 'legacy' features (if you can find them), but overlaying it all with a meretriciously elegant but largely un-navigable visual environment which always occupies the full screen. This major 'controlling' part of the system now runs 'full-screen' with no scaling or overlapping. The reduction in the user's ability to flip through and find things he or she needs is wholly retrograde. A desktop computer driven by mouse and keyboard is not a tablet or a phone and the visual and control metaphors just don't work in the same way. Windows 8 badly needs a 'Professional' mode which would reinstate the more familiar appearance and functionality for power and professional users. IT people and computer enthusiasts will get to grips with it, but it's unlikely that will happen in a Corporate environment for a few years - at least not until the domestic take-up from 26 October has enabled a large number of people to become familiar with it in their own time.

Windows 8 'Start Screen'

As an example, having today started the Microsoft Update process in a 'Modern' (formerly 'Metro') window, it was then almost impossible to find the other applications to carry on working with them or to flip back and check on progress with the updating process. We installed a second monitor on our test machine which allows us to have 'Modern' in one window and 'Desktop' and applications in another which helps a bit, but it is still very confusing to use, and there is no way easily to find and start applications, especially if, like most people, you have installed a lot of them.

Expect to see a plethora of third-party software soon which claim to reinstate something like the 'Start' menu or which provide a variety of patented 'shells' to make the navigation easier. We used to see a lot of these at one time, and then the standardised Windows Start menu became good enough not to have to bother with them any more. I've re-discovered the 'Window-R' command button sequence and have been using that to launch a program by typing its name into the box..... and 'Alt-F4'  to close it - which is going back 25 years in terms of usability to the days when I taught DOS 6.2 to excited classes!

Perhaps Microsoft will introduce a 'conventional' interface later as an enhancement. I do hope so.

If you are offered Windows 8 on a new computer from October 2012, and as things are now, I'd strongly recommend that you ask the supplier very politely and firmly for a retrofit of Windows 7 before you take it away. If you want the admittedly superb performance improvements in Windows 8, you could install Start W8  (which is very simple. similar to the Windows 7 start menu) and  Start Menu X (aka Vista Start Menu) or Stardock and regain a useful desktop launcher that way. A useful 'How to' article offers more ideas.

The 'Windows Secrets' newsletter offers some simple and effective advice on getting started with the new 'Modern' interface version of Windows 8.

---

Microsoft Office 2013 Applications
Another serious loss in the new Microsoft 'Modern' world is the new 'un-textured' appearance of standard Microsoft applications which have exchanged their subtle grey/blue tones and changes in visual emphasis around the edges for a new stark un-textured all-over white. The former design was elegant and gave the programs a sense of relief and space, making them easy on the eye and a pleasure to navigate. The new designs are visually tiring to work with.

New 'minimal' MS Office interface

---

Windows 8 and Windows RT compatibility and DVDs
As a Postscript to the above, there will be considerable confusion from 26 October 2012 between the various versions of Windows 8 - which includes Windows 8 RT for tablets. Woody Leonhard has written a helpful summary of the differences, but it is going to be very confusing indeed. And you won't be able to play DVDs in Windows 8 either unless you buy a specific media-enabled version or add-in. There's a useful summary video on C|Net which outlines the new interface:

• Comments [2]

Distributing software licenses over the web for T189

David Clover  2 October 2012 09:49:23

My colleague Jef has just delivered our latest utility which is designed to allocate software license codes to registered students over the web using an IBM Domino XPage front-end.


Collecting the license
Students registered on the Digital Photography (T189) course can visit the website, login using their Open University account details and then collect a license number for the Adobe Lightroom software which they need as part of their course. To make things tricky for them, there are two versions of the software to choose from depending on which computer platform they use, so the system suggests the most appropriate option for the detected computer type and then allows them to choose the exact version they need.

Management Information Interface
At the same time, an XPage-driven management information interface provides details to the T189 course managers which enables them to report the allocation of licenses to the software supplier for formal registration via a spreadsheet which is generated by the web page on request.

Web Service
The back-end programming for this involves an invisible 'behind-the-scenes' call to a University Web Service which tests the user's login ID and makes sure that the student is paid-up and properly registered. If the check is successful, the student proceeds to the selection and allocation screen.

IBM XPages
We've used IBM's XPages environment for this, and created a simple-to-use web-delivered 'Management' interface for the Faculty staff involved in running the course. This allows them to monitor progress of students in collecting the software licenses and allows them to resolve any queries. Once again, the IBM XPages environment has proved more than capable of supplying a superb and secure rapid-development environment with great flexibility and power to turn what was a very complex and time-consuming manual task into a simple and efficient computer-driven one.

Confirmatory Email
Once the user has collected his or her license code, an email is created which automatically routes a confirmation to the registered email address of the student as recorded centrally by the University using our in-house 'CAMEL' utility which is designed to facilitate and automate efficient mail contact with students.

• Comments [0]

Appying OUICE Templates

David Clover  11 August 2012 11:47:08

We've recently applied OUICE templates to three websites which were incompatible with the newer University design standards. These had an older 'look and feel' and need our new OUICE design.

These include our 'Safe Computing' website which has had a makeover in the Druipal PHP framework and our MCT Faculty website which is the public face of our Faculty area of the University. Our IBM Domino developer colleagues John, Paul and Colin at Online Solutions Ltd have enhanced the MCT Faculty site for us in a very short time, and it's been interesting to compare the outcomes.

---

MCT Faculty Website (IBM Domino environment)

---

---

Safe Computing Website (Drupal 7 environment)

---

---

ITD Wiki Website (IBM Domino 'Dominowiki' environment)

---

---

MST209 Website ('Flat' HTML site)

---

In the case of the University's Drupal distribution, the OUICE look and feel is supplied as a theme, and thanks to the excellent craftsmanship of my programmer colleague Jacques, we've successfully modified and mounted this onto the much improved Drupal 7 environment for the Safe Computing site. In the process Jacques has managed to work out most of the tecnhnical and design issues in what is a very complex development framework and the process has been an interesting and worthwhile one.

But the flexibility of the OUICE theme means that we have also been able to apply it to several legacy websites which needed a makeover, two in the IBM Domino environment and one in a flat HTML site.

Our IT Development Wiki uses the DominoWiki template first released in 2004 which we acquired via OpenNTF from Ben Poole of the London Developer Co-operative. This has a very good approach to CSS management which Jacques has managed to reverse engineer very effectively. The MST209 site started life in a Dreamweaver environment and now, again with Jacques assistance and in-depth knowledge of OUICE has the new-style template wrapped around it.

So our plan now is to move to 'look and feel' compliance with the University's new standards without necessarily having to discard or compromise our investment in valued technologies such as IBM Domino which are so beautifully straightforward to manage and update. And our next move will be to start exploiting the powerful IBM XPages environment in Domino/Notes 8.5.3 with similar visual updates as and when we can. In particular, because the Domino Rapid Application Development (RAD) technology is mature and very much database-oriented, it allows us to bring forward and represent data from many various sources in an organised and consistent way.

• Comments [0]

Collecting money on-line for events

David Clover  26 April 2012 16:03:16

All large organisations are able to collect money securely from a credit or debit card online as part of a booking or purchasing process and The Open University is no exception. We now have a Faculty utility that allows people organising conferences and events to collect sign-up information from potential delegates and clients, and then pass them online through to WorldPay via a University-designed secure API where they can make credit or debit card transactions safely.

This means that staff in various departments who are organising events don't have to engage with insecure 'Cardholder not present' telephone conversations about taking credit card details and those signing up can get immediate confirmation of their acceptance onto the event. The system makes collecting money and participant details efficient and cost-effective.

We have so far provided 5 separate environments for collecting fees for conferences and events and it's working smoothly. At the time of writing the system has collected £14,000.

Here's an example of an online form that's been used recently for an event in the Mathematics and Statistics Department.:

---

---

We are now putting the final touches to an information module (shown above) so that organisers can download the most recent list for their event as an Excel sheet. They can then use it to create lists, labels, mail and email merges to help them to manage the event itself.

• Comments [0]