Oxford University’s Gardens, Libraries and Museums (GLAM) engaged CirrusHQ to create an AWS-hosted solution with a fully automated Continuous Integration / Continuous Delivery (CI/CD) pipeline
Executive Summary
The University of Oxford has four museums containing some of the world’s most significant collections. Not only are these important for scholarly research, they also represent the front door to the wealth of knowledge and research curated by the University. The goal of the University’s museums and IT Services team was to make the detailed data it holds for these collections accessible – and searchable – online from a single location.
One of the key technology challenges related to this was the creation of an infrastructure able to respond, perform, and deliver accurate search results from the data sets. In the process of bringing the University’s collections under one digital ‘roof’, the in-house development team would be onboarding new data sets on an ongoing basis and they wanted to build up a set of functions able to interrogate existing data from the same single search tool as they did so.
The solution CirrusHQ provided for them utilises managed services by AWS to maximise performance and reliability and features a fully automated Continuous Integration / Continuous Delivery (CI/CD) pipeline. This enabled the development team to initiate builds across the environments, with suitable governance controls and points for testing these updates.
………………………………………………………………………………………………………………………………………………………………………………………………………
Speak to an expert – Contact us for a no obligations discussion to review your requirements and we will have the right expertise to guide you.
Latest paper – Download “5 key reasons your organisation already needs the cloud”.
………………………………………………………………………………………………………………………………………………………………………………………………………
Business Challenge / Problem Detail
GLAM and IT Services wanted each museum’s collection to be viewed as a discrete entity, but they also planned to create a single mechanism facilitating the online search and discovery of items across the University’s collections. The solution would be a key resource allowing scholars, students and the general public to interrogate these world-famous collections online and it was important to ensure prompt (and appropriate) responses from the searches.
The team at GLAM and IT Services have the in-house skills and resources to build the solution’s core functionality, taking the API calls from the website, gathering the data, and sending the responses and any assets associated with the request. However, they recognised the advantages of deploying this application on AWS’s managed cloud services and taking advantage of its serverless technology.
The team recognised that the solution needed to be secure and robust, with capabilities for backup and restoration. In addition, the team required continuous integration and deployment of the solution that could be managed internally. Finally, they wanted to ensure that the solution provided GLAM and IT Services with a blueprint for best practice and was cost-effective.
Solution
The AWS solution that CirrusHQ provided was created using AWS best practices and utilising AWS Well Architected Framework.
CirrusHQ built the AWS infrastructure that is shown above. CirrusHQ ensured that all aspects of the infrastructure could be controlled via code using AWS CloudFormation. All source code for both application and infrastructure is managed within the GLAM and IT Services GitHub account.
The diagram above shows, at a logical level, the services used and all application level calls to and from the solution. The use of managed services provided by AWS for this solution ensures best practice, high availability, scale, resilience, and maintainability of each component.
The API gateway was used a secure front door (along with AWS WAF) to the solution, taking advantage of AWS’s fully managed service to create, maintain, monitor, and secure APIs at any scale.
Working with the in-house development team, AWS Lambda was used to invoke each search request from the API separately. This ensured execution times, therefore reducing costs, and performance are maintained. In addition, separating the functions into their logical parts to better allow more targeted builds and updates, plus benefiting further-proofing the solution.
One of the key deliverables of the project was to ensure that the development team had the ability to control integration and delivery of new features to the targeted environment and accelerate delivering new features and improvements to the system. GLAM and IT Services use GitHub to manage their code; but in addition, they wanted to have the ability to include additional governance when releasing to controlled environments (production). CirrusHQ used AWS CodePipeline as a tool to implement CI/CD. This included creating a pipeline builder for each pipeline for the targeted environment so as to make the CI/CD pipelines themselves a repeatable and immutable part of the solution as well.
To deliver the new workload on AWS, CirrusHQ worked with various members of the University of Oxford team, including developers, infrastructure architects, project management, IT support, business analysts and business representatives. They built out the AWS infrastructure environment in a repeatable format, creating an automated process allowing the team to build and deploy on AWS regularly on a continual integration and delivery basis.
Outcomes
Overall, the solution not only enables high performance and resilience from the adoption of resources that are managed by AWS, but it is also cost-effective. As all the infrastructure is in code, it is repeatable, easy to update, and has a delivery pipeline that is able to be managed by the in-house development team to ensure continued updates are quick, and do not constrain their development or deployment processes.
The solution has enabled the GLAM and IT service team to have an AWS blueprint for best practice, demonstrating use of serverless, code pipelines and services managed by AWS to gain the most from current practices within the public cloud.
………………………………………………………………………………………………………………………………………………………………………………………………………
Please visit the Oxford University Museum of Natural History website to see the new collections online search functionality in action.
Take the next step in your cloud journey – Contact CirrusHQ for a no obligations discussion to review your requirements and we will have the right expertise to guide you. Wherever you are on your cloud journey our end-to-end expertise enables us to deliver the right balance of advice, development and affordability that works for you – to improve your services, performance, flexibility, robustness, security and cost.
………………………………………………………………………………………………………………………………………………………………………………………………………
Tooling (Tools and Technologies)
Developer tools: GitHub, AWS CodeBuild, AWS CodePipeline, AWS X-Ray
Management and Governance: Amazon CloudWatch, AWS CloudFormation, AWS CloudTrail, AWS Config, AWS Systems Manager
Security: AWS Identity & Access Management, AWS Key Management Service, AWS WAF
Application Integration: Amazon Simple Queue Service, Amazon Simple Notification Service
Compute: AWS Lambda, Amazon EC2
Database: Amazon ElastiCache
Storage: Amazon Simple Storage Solution, Amazon Elastic Block Store
Networking and Content Delivery: Amazon VPC, Amazon API Gateway, Amazon Route53
Quotes from the University of Oxford
Professor Paul Smith
Director of Oxford University Museum of Natural History
https://oumnh.ox.ac.uk/people/paul-smith
“We have seven million specimens, and we have a fraction of those available digitally. And like many natural science collections, we have about 10% of our material digitised at some level or other on a much smaller percentage available online.
Online collections enables our researchers to assess whether the specimen that they’re looking for is the right one – rather having to travel to Oxford to see these or transport it to the research institution concerned”
Kevin Rodd
Divisional Secretary and Chief Operating Officer
https://www.glam.ox.ac.uk/kevin-rodd
“A key objective of GLAM is to share the University’s collections to the public and researchers all around the world. The diversity of collections within GLAM enables connectivity across these different collections which brings additional benefits and insights.
Digitisation provides us with a great opportunity to open our doors virtually to the world and this is the first step in that journey to making the collections accessible to the world. There is also potential for financial gains through this channel by engaging with donors or developing a picture library.”
Dr. Sarah Joomun
Digital Collections Manager
https://oumnh.ox.ac.uk/people/dr-sarah-joomun
“Previously; it was almost impossible to find information about our collections and what we actually had in the museum. So this project was the chance for us to actually set up a single search for all our collections and a single place to look for collections data, display this and add additional information the collections.
For the first time, we have got a really good search service with images. You can search across all the collections. Previously, if you want to find out about everything we have from a particular country; you had to search about 20 times now you have just search in one place and all the data comes up at once. Now you can search across our fossils and insects at the same time.
Our visitor have started sending enquiries to museum which are getting more specific which is indicative of the fact that visitors will have looked on the Collections Online, being able to find information online and then asking questions based on that.”
Anjanesh Babu
Systems Architect and Network Manager – Gardens and Museum (GLAM) IT.
“The level of expertise in solution design and support provided by Cirrus HQ has been a formative experience with this project for Garden and Museums IT. Having a simple – yet robust support framework means setting up our AWS operational infrastructure has been seamless.”
