We made the strategic decision for the VQ Conference Manager 3 platform to move all logging and data capture into the Elasticsearch database and provide visualization via Elastic’s Kibana tooling.
We’re now about 6 months into having VQ Conference Manager 3 deployed in the field and I’m incredibly pleased (and relieved) by the massively positive feedback on the decision; I have been amazed how many customers (and potential customers) have said they’re already using Elasticsearch and Kibana within their organization.
Our initial goal with moving to Elasticsearch and Kibana was to use best of breed, industry standard, tooling to capture and enable the visualization of logging and reporting data coming out of VQ Conference Manager 3. We are so committed on the decision to include Elastic as part of VQ Conference Manager that we signed up to become an OEM partner of Elastic; this involved a fairly substantial $ spend over the next three years – it does mean, however, we get support from their support teams and have been able to resolve problems quickly.
I have to admit though that there were times during the VQ Conference Manager 3 development process I worried we’d made a mistake when we were fighting with a whole raft of issues and nothing seemed to be working; there was a (quite long) period where we seemed to go backwards more than we moved forwards. However, as things started to stabilize and I started to understand how Elastic worked (and we had updated the VQ Conference Manager core to start generating the appropriate data), we started to put queries, visualizations and dashboards together that yielded results that were way in excess of our expectations, providing analysis and insight into issues that would have required huge amounts of manual work in previous versions of the product. We were able to reduce complex (and apparently random) issues down to easily digestible graphs; the problems became defined, well understood and from that, resolvable. From that point on, I was a fully paid-up member of the “Elasticsearch and Kibana is awesome society”.
Building in that experience, we moved forward (at this point, VQ Conference Manager 3 was starting to work reliably – again, I’m sure you can imagine this but that was quite a relief. I started sleeping again) and expanded out the set of Dashboards available in 3.0; as with the internal analysis, the insight we gathered into calling patterns, what the system was doing etc. was way beyond what we’d been able to do in previous versions of Acano Manager. Love blossomed.
VQ Conference Manager 3 is based on a really powerful set of technologies – Kubernetes and a concept called “Containers”. Containers are the things that contain the software components that do the work; Kubernetes is the thing that makes them all together (the so-called orchestration layer). So, in VQ Conference Manager 3, we have a whole bunch of containers – some containing our VQ Conference Manager software and others containing things like the database, Elasticsearch and Kibana. The brilliance is that we can take off the shelf Containers and host them with the VQ Conference Manager service orchestrated by Kubernetes. Each container is, essentially, its own virtual machine; we can, therefore, run different Containers and not have to worry that different component dependencies interfere with each other resulting in obscure system failures. Each container is isolated and runs as a well-defined black-box.
So, to summarize the ramble so far: VQ Conference Manager uses state of the art open source technology (Containers and Kubernetes) to allow us to package software into a solution. The VQ Conference Manager 3 solution contains Elasticsearch and Kibana.
The guys and gals at Elastic pay for all of their brilliance by cleverly separating out the functionality that customers value and making them available as an optional extra; the optional extras used to be called X-Pack and are now called “Elastic Stack”.
We haven’t focused too much on X-Pack/Elastic Stack up until now because everybody is getting used to and familiar with the power and greatness that’s available in the base version. Our higher priority has been making some changes in how the VQ Conference Manager 3 platform is generating data (and extending the data) to make it easier to generate visualizations and address some of the feedback (for example, making the call duration available in minutes or hours, adding cost codes) we’ve received (or concluded based on our own experience).
As an Elastic OEM partner, you can buy Elastic Stack from VQ to work with your VQ Conference Manager solution.
The following is the list of things it enables that we think are appropriate and useful at the moment:
- The ability to export the data from reports as a csv
- PDF export of reports (see attached example)
- The ability to set threshold values of specific data values within the data and send emails, post messages to Slack or inject data/messages into Elastic
✔ Elastic SQL
- Run SQL queries against Elasticsearch
✔ From Elastic 6.5 (VQ Conference Manager 3.1 will run at Elastic 6.4)
- Cross-Cluster replication (beta). This will become a really powerful tool – it’ll allow, for example, data from one Elastic Cluster to be replicated to another. Usage scenarios include backing up data or have dedicated “analysis” hosts with extra capacity.
- As VQ Conference Manager 3 based systems become bigger, this will become invaluable.
✔ Other features enabled include:
- Graph – the ability to establish relationships between data (example use cases include fraud detection and malicious system access)
- Machine Learning – a whole bunch of coolness
- Canvas – a next-generation visualization tool
What is striking about Elastic and Kibana is the velocity of innovation; new releases are frequent and rate of great new functionality being added is amazing.
We are very pleased to have added Elastic and Kibana into the VQ Conference Manager solution.
The process of buying Elastic Stack is straightforward; talk to VQ sales and raise a PO. We’ll issue you a license key which you upload into Elastic; job done.