Citrix released another CVE Vulnerability announcement last week affecting all currently supported of ADC (Netscalers). https://support.citrix.com/article/CTX281474
The vulnerabilities are addressed in the following supported versions:
- Citrix ADC and Citrix Gateway 13.0-64.35 and later releases
- Citrix ADC and NetScaler Gateway 12.1-58.15 and later releases
- Citrix ADC 12.1-FIPS 12.1-55.187 and later releases
- Citrix ADC and NetScaler Gateway 11.1-65.12 and later releases
- Citrix SD-WAN WANOP 11.2.1a and later releases
- Citrix SD-WAN WANOP 11.1.2a and later releases
- Citrix SD-WAN WANOP 11.0.3f and later releases
- Citrix SD-WAN WANOP 10.2.7b and later releases
Over the last week we patched 24 ADC pairs 11.1, 12.1 and 13.0 to ensure we were unaffected. However there were a few issues along the way.
In patched ADC version 11.1 -65.12 there is a bug affecting DUO 2 factor authentication prompts where the screen which stick at “Success! logging you in” screen. This issue does affect all users all the time and a browser refresh or different browser may work. However, Citrix technical support confirmed a bug as other customers were having similar issues. The only workaround is to upgrade to 12.1-58.15 or 13.0-64.35.
The smoothest ADC firmware patch was to 12.1-58.15 where no issues were reported.
ADC 13.0-64.35 Firmware Update & Traffic Policies
Something you must do in ADC 13.0.-64.35 is add a traffic policy to your Virtual Gateway if you are logging for Citrix VAD and Storefront access.
The result is without a new traffic policy is Storefront throws a “Can’t complete your request” message. Nothing anyone wants to see.
Adding a traffic policy CLI
add vpn TrafficAction Traffic-SSO-action http -SSO ON
add vpn TrafficPolicy Traffic-action true Traffic-SSO-action
bind vpn vserver _XD_MYGATEWAY -policy Traffic-action -priority 100 -gotoPriorityExpression END -type REQUEST
Replace MYGATEWAY with your Virtual Gateway name. There is no public documentation for this yet, I’m told by Citrix Support and its a new Security feature.
How we did it!
Our Citrix ADC VPX Appliances are all in high availability pairs so 24 pairs is really 48 Netscalers, includes some DEV and UAT ones. Don’t forget that even a UAT or DEV Netscalers could be a backdoor to your if not patched.
It would have taken a very long time to upgrade each HA pair manually and so we utilized the powerful ADM (NMAS) maintenance feature to push out the updates. But we still supervised the processed and of course tested once the upgrades were completed. I will produce a video on this process and include it here.
Basically, you create a Maintenance Job with-in ADM
Networks > Configuration Jobs > Maintenance Jobs
- Click create Job
- Upgrade Citrix ADC/ADC HA
- Give the Job a name
- Add ADC instances
- This will do some prechecks to ensure no disk space issues
- Select the image (Download from Citrix)
- Then you can choose to schedule it for later or run it now
- If configured select Email the report. Then you will know when it finishes
Because we graphically all over the world some regions will take longer than others. In our case we host our ADM in our datacenters. Citrix Cloud ADM which work differently.
More videos on this process will be updated shortly, please consider subscribing to my channel.