David Clover


    Using Domino IMAP with Outlook 2013

    David Clover  29 March 2014 12:02:40
    Long-standing IBM Domino/Notes users (and probably CEOs keen to spend a vast sum of money getting rid of Notes in favour of more complex, expensive and inflexible workflow systems) often complain that 'you can't use Outlook if you have Notes'. It's not true, though there will always be an issue with calendar functions as things stand now. You can't integrate them as you can with an Exchange server.

    We have an IMAP service running on our IBM Domino 9.0.1 server (and we've had it on all previous versions) and it works pretty well. I have used Domino's IMAP protocol successfully with MS Outlook with all versions since Office 2003, and possibly earlier, with success.

    However, we have some bad news when using IMAP with MS Outlook version 2013. In this version, although the Domino Mail 'Trash' folder is detected and included in the folder tree, it turns out that you can't specify Domino's IMAP 'Trash' folder in Outlook 2013 for deleted items. Outlook 2013 creates its own called 'Deleted Items' which means that you have to keep on dipping back to the full desktop Notes client (or INotes web client) to empty the Trash. In fact you first have to move the items in 'Deleted Items' in the Notes client and then use the 'Empty Trash' command to actually get rid of the messages.

    The Mozilla Thunderbird client seems fine - it sensibly regards the 'Trash' folder on the server as the 'Deleted' folder. Outlook 2013 doesn't so this though. It's a a retrograde step for Outlook 2013 as it will affect connection to other mail servers too - there has been discussion in a Micrsoft forum about this bad behaviour in Outlook 2013

      Forward to the past - Word for Windows 1.1a

      David Clover  28 March 2014 15:43:45
      I was astonished to see that in my last few days working here (or probably anywhere), the original source code for Word for Windows 1.1a (1989) has just been released for download. This won't of course give you a free copy to use - it is just the original program code that was compiled into the commercially sold program.

      By 1989 I was a veteran of Word for DOS (released 1983) which we used (and which I had taught people to use) from 1987 onwards. In particular, we made substantial use of Word's excellent 'Style Sheets' to achieve structured authoring of our many teaching documents and manuals. You can see more about the early DOS Word at ToastyTech. It was well ahead of the earlier software we had used 'WordMarc' (aka 'MUSE').

      The arrival of Word for Windows 1.1a was eagerly anticipated and we were not disappointed. In particular its automation features using a macro language and 'Fields' now collected and still eminently usable under Word's 'Quick Parts' label allowed undreamed of flexibility in designing documents and workflow.

      word_for_windows_1.1a_2 style=

        Further developments in Windows 8

        David Clover  27 February 2014 13:09:56
        I've mentioned before that we have experimented and indeed use Windows 8 for some machines. As I also wrote before you can readily tweak Windows 8 so as to bypass the cumbersome 'Metro/Modern' interface (if you are a desktop user) to the point where it works in virtually the same way as Windows 7.

        Recent Article

        A recent article by the excellent 'Windows Secrets' author Woody Leonhard provides interesting reflections on the whole Windows 8 issue and how Microsoft have handled and responded to it.

        Windows 8 Development Staff

        The key staff responsible for the Windows 8 development are no longer working for Microsoft and this is provided in detail in Woody's article.

        My recommendation

        I continue to believe that Windows 8 is a perfectly effective product for desktop users as long as you bypass the new Metro/Modern navigation features and install a backwards compatible start menu.Then you can take your pick of whatever new features you might want to experiment with 'on the side' of your 'real' work through the 'Charms' pop-up bar.

          Applying thousands of redirection rules to inbound messages in Domino

          David Clover  17 December 2013 13:02:57
          We have a system which sends out emails when there's an issue with responses to the monitoring of assessment activity related to students and tutors. A few years ago, we created a very simple but effective rules parser in our IBM Domino system based on a standard mail template. This looks at around 6100 rules relating to the content of the subject in the email (there is a small number of message types but these have to be related to other criteria as well which means 39 rules for each course title across 13 possible centres) and then resends the message to the appropriate place.
          Image:Applying thousands of redirection rules to inbound messages in Domino

          Now we have to adjust this system slightly because we have new types of contracts relating to the monitoring activity and therefore we need to create supplementary rules as to where the diagnostic messages should be sent in future.

          So we are going to add a new table and design an algorithm in conjunction with our internal client for a secondary look-up to make sure these messages go to the correct person in our 'new' contractual environment.

          Because of the superb flexibility inherent in the Domino system we are hoping to install and test this in a working day.

          I'm really looking forward to seeing what our team can do with this one.

            Concatenating multiple files into a PDF

            David Clover  18 October 2013 09:37:40
            Our students will soon have to submit all their Electronic Tutor Marked Assignments (ETMAs) in a single PDF file for marking, but there is a problem for those, for example in the Maths and Statistics areas and others, whose ETMAs have to consist of a heterogenous variety of original file types - .The file formats that we can convert to pdf and then merge include: doc, docx, pdf, jpg", "jp2", "jpf", "gif", "png", "tiff", "tif" objects in various formats. The new requirement is for these to be presented for assessment as a single concatenated PDF file.

            My colleague Henryk is putting the finishing touches to an Apache Tika processing tool to detect content types which then feeds into a utility from Qoppa software driven by Java via our Domino server. Students will be able to submit a .ZIP file containing all the disparate objects which are uploaded and after Tika's filetype detection, processed by Qoppa into a single .PDF file which students can then submit in the usual way.

            We have spent some time attempting to develop an IBM Domino XPages tool for this, but because of incompatibilities in processing .docx files between the Qoppa back-end environment that does the processing and the XPages environment we have had to re-think and have returned to 'conventional' web input techniques using LotusScript and Formulas. Having now solved some Java runtime permission issues on the server, this has now been shown to work extremely well, and so it just remains to build and tune the web front end in order to deliver the system to our students.

            The workflow concept is that a student will authenticate with the web page as usual, and then upload the ZIP file. After a short wait, during which the protocol conversions are done on the server in the background, he or she will be presented with a web page with the uploaded .ZIP and the newly concatenated .PDF file. This can then be downloaded and passed across to the University's ETMA system in the usual and approved way.

              Office Schedule tool in XPages written in two days

              David Clover  12 September 2013 10:10:38
               A neighbouring department had an urgent need for a utility to meet a work planning need. The office supports a wide range of activities across the organisation. To date, the office had been using a shared MS Excel spreadsheet to manage work scheduling, and this approach had been used for several years despite a number of disadvantages – only one person could use the spreadsheet at a time, records could be accidentally over-written or deleted with no audit trail and the amount of data that could be displayed was limited. Excel is not a suitable tool for a task of this complexity. The schedule also employed a complex system of colour coding to indicate priority, status and due date, and these colours all needed to be applied by hand where relevant: a time-consuming manual task.  Despite these limitations, this Excel schedule was the main record of ongoing, upcoming and completed work for the whole survey office  team and as a result was integral to the office’s smooth running.

              While moving to a web-based tool for managing their scheduling has been a desirable idea for some time, it became a higher priority following an incident where the office found that a record had been accidentally deleted, causing inconvenience to a client who was expecting some results and considerable disruption to the office as work had to be hastily re-arranged to prioritise work that had been missed.

              So an IBM Domino web-based tool was created by a member of our MCT IT-Development Group running under XPages, and because of the urgency, it was developed in two days of intensive programming and analysis.

              The main benefits of the new approach are:
              • Everyone can use the schedule at the same time;
              • It’s impossible to delete a record by accident;
              • ‘Info’ entries (the purple bars across the whole sheet) will automatically disappear once their date has passed;
              • The client group will be able to record more details about jobs than are currently possible – a notes field has been added which can hold an unlimited amount of text;
              • Each member of the client group has a personal view so each person can see just those jobs that are relevant to him or her;
              • People outside the survey office, including managers, can be given access to use the link to see jobs in progress without needing access to server folder shared area;
              • The green/blue shading for this/next week’s jobs will be applied automatically over time rather than needing manual updating;
              • Jobs will automatically move to the ‘finished’ tab when they have been completed.
              The client department is thrilled with the new environment and it represents another significant achievement for the rapid development potential and flexibility of our Domino 9 XPages rapid development environment.

              Here's a screen shot of what the user sees:
              Image:Office Schedule tool in XPages written in two days

                Allowing users to update personal profiles

                David Clover  9 September 2013 11:13:00
                For many years - since 2004 in fact - we have been showing 'personal' pro-forma details in the context of departmental websites using a back-end Domino database. But up to now, updating those details has only been possible with an IBM Notes client, and that's been restricted to those people responsible for content updating in various departments.

                Now that is about to change for the Faculty with the imminent release of a web-based interface at - this will allow a user to login and then see and edit their own personal details directly.

                The information is stored in our tried and trusted IBM Domino database (which we call 'MaCuNames' - it drives a number of Faculty 'people' information systems) but the entry about a person can now be extracted and displayed on any web page served from the campus using a server-side XML request with a specifically tailored URL. Campus users can see such an extract for my own details and possibly their own by altering the URL slightly - David Clover

                This means that we can make a server-side request from our Drupal environment to populate a page like my own in the new Faculty website, and we will also be able to populate any other site that can make a similar server-side call to present the same data in whatever context is needed. We have designed it so that it is backwards compatible with older websites which use IBM Domino as the presentation engine without any changes.

                As far as editing the personal content is concerned, we provide three 'slots' for users to populate. These are 'Biography', 'Research' and 'Teaching'. If a particular 'slot' is not filled in, the related tab won't appear on the presented public web page. In my own example case I have completed all three, but in the case of others, for example my colleague Blaine Price in the Computing and Communications Department only the 'Research' tab is visible. The other tabs - 'News' and 'Publications' will appear if the person represented has any news items or research publications recorded against their name. 'News' relates to our own 'News' module (also an IBM Domino back end source with a web-editing front end) and our 'Open Research Online' database operated by the Open University library. It's also possible to email the person via a web form- which ensures that spam harvesters can't scan the page for an email address. The web form is protected with a 'Captcha' module. We've also included links to other 'Social' aspects for the user -including Facebook, Twitter, RSS feeds from blogs, personal web areas and so on.

                Some of the information relating to a person is imported daily from the University's HR system, and some is held only in the Faculty. Users can modify the details directly where they are held in the back-end IBM Domino database, but where the information is imported from elsewhere, we provide outward links to the University's HR web-editing sites where they can modify their own data for overnight automated import to our own environment. We have also provided an option whereby people can opt not to have their details selected for websites at all if they want to remain anonymous.

                There are plans afoot to provide a University-wide service for holding 'people profile' data but it's not likely to be ready for some time. When it is, we expect to be able to import from it to our own system (or indeed export from our system to it as the case may be) to provide a standard and coherent statement visible wherever it is needed on a website. Examples of the input screens are shown below.

                Image:Allowing users to update personal profilesImage:Allowing users to update personal profiles

                Image:Allowing users to update personal profilesImage:Allowing users to update personal profiles

                Image:Allowing users to update personal profilesImage:Allowing users to update personal profiles  

                  Updating Faculty and Department Websites

                  David Clover  25 June 2013 23:38:59
                  Our current and major task, and certainly not a trivial one, is to provide a new 'focal point' website to reflect major organisational changes that have just taken place in our Faculty of Mathematics, Computing and Technology.

                  This had originally been planned for October 2013 but was brought forward to August 1st 2013 and so this gave us very little time to build the environment. In fact the services on the new PHP sites referred to below didn't exist a month or so ago.

                  The 'Presentation' layer uses a popular bulletin board PHP framework (Drupal), but large sections of it are populated dynamically using XML and Web Services from our existing IBM Domino content containers. These containers also service other existing and 'legacy' websites and services which are presented directly from the Domino http server - sites such as Centre for Research in Computing and sites such as our Faculty intranet for example

                  We have described the plan in outline for (in-house readers) in a PDF file.

                  We have been instructed that many of our team's (and indeed users') long-held assumptions about how we represent the Faculty and its three new departments via a web information system have to be put aside, so we are, to a certain extent, making it up as we go along.

                  What is clear though is that it is very easy to exploit the mature XML techniques available from our existing IBM Domino databases and infrastructure to provide fast and efficient web services so that we can capitalise on the data containers we already have for Staff and research projects, news and events. We can do this in a way that allows us to re-present single sources of data within the PHP environment via XML and linked data. For some examples see these pages: We have done the same with news items at: From a development point of view, it would have been very much simpler for the team to have devised everything within a single Domino data and presentation framework, but the Web Services, XML, Linked Data and SOA elements that we can expose from our back-end Domino containers are the key to flexibility in commercially focused organisations and we can use them too. We should be able to extend this seamlessly as needed, and it will extend greatly the technical range of our team which is now well-versed in the existing and simple to manage IBM Domino 9 environment and increasingly the PHP framework we have been instructed to use.

                  As ever with web-based environments, especially one created very quickly like this, the technical challenges are more than matched by the even larger challenge in getting meaningful content into the new site; we will probably be plundering our 816 or so existing Faculty sites for suitable content in the short term.

                  We will offer staff working as editors and content providers with an Active Directory login via LDAP to the new site for editing and they will use 'in place' editing of static PHP content. We'll also be providing web-enabled editing of the dynamic/SOA content using Domino XPage interfaces such as which we are just finishing now.

                    Collecting Tutorial Attendance information

                    David Clover  19 June 2013 16:34:20
                    Every year our hardworking Associate Lecturers carry out a large number of face-to-face tutorials all around the country. This is how we bring the 'local' element into the distance teaching we do as a national University specialising in Supported Open Learning.

                    We need to collect statistics about how many students attended the face to face sessions, and to record for onward report to the University how many scripts were received from the group that was tutored.

                    Elsewhere in the University this troublesome chore is carried out in a very 'manual' way, perhaps by filling in and concatenating spreadsheets and netting them up laboriously into summary reports. However, for some years, our Faculty of Mathematics, Computing and Technology has been collecting this electronically using our IBM Domino server, and this year, we completely re-built the system using IBM Domino XPages. This automated collector is now up and running very successfully. So far we have collected nearly 2000 records from our Faculty's Associate Lecturers.

                    The IBM Domino system, developed by my colleague Jef, makes extensive use of XPage controls; an example is shown below. We also use a combination of Active Directory and Web SSL login cookie detection techniques to collect some of the details of the tutor completing the forms. This means that each sees only his or her own entries and only validated users can make entries and changes. Once they have authenticated, we can allow them to edit and delete their own entries too. Using Domino's useful ACL 'Roles' we can quickly assign or emove extra privileges to those who administer the system so that they can see all the entries from all users in the same web interface.

                    Another XPage control allows us to upload the entire collection of reports, or part of it to an Excel sheet. We can also show as tooltips in the list of items any comments that have been placed on the underlying document. You can see one of these in the sample below. A simple workflow routine checks that the entries placed on the form are complete and consistent and advises the user of anything that might be missing before saving,

                    Once again, IBM Domino and XPages has provided elegantly and quickly (in one easily managed Domino database object) a solution that would have taken much longer to develop using alternative techniques.
                    Image:Collecting Tutorial Attendance information

                    Fig 1: Log Display for a user

                    Image:Collecting Tutorial Attendance information

                    Fig 2: creating a new entry.

                    Image:Collecting Tutorial Attendance information

                    Fig 2: Offering an option to edit or delete an existing record

                      Quick Survey Requirement met using XPages

                      David Clover  26 April 2013 12:11:44
                      We were asked to put together quickly a survey form which will be used in-house to collect details relating to 'External Events' requesting the levels of support required for them.

                      The survey form had a number of 'conditional' elements - some dialogue boxes were to appear only when certain checkbox options were chosen by the respondent. Some single questions had single 'radio button' choices and some single questions had a requirement for multiple check boxes to indicate preferences. The whole had to be secure and visible to a respondent only after logging in to our Campus authentication system (which we call SAMS) and it had to recognise the user to allow the form-filling to proceed.

                      My colleague, Jef Lay, assembled an suitable form within a day using standard XPages controls. The form will collect the data and a separate view just accessible to the requestor will show responses and allow download of the responses into an Excel format for subsequent manipulation.

                      There was no comparable option available at short notice and once again we've proved how efficient and versatile the Domino XPages model is when it comes to working with web-enabled environments.

                      Here's a snapshot of the form which is also visible at our Website to internal staff.

                      Image:Quick Survey Requirement met using XPages

                      Safe Computing

                      © Faculty of Mathematics, Computing and Technology at the Open University Copyright
                      The Open University is incorporated by Royal Charter (RC 000391), an exempt charity in England & Wales and a charity registered in Scotland (SC 038302).