The ERIS Infrastructure team will perform maintenance on mysql.dipr.partners.org, mysql2.dipr.partners.org and mysql3.research.partners.org on Saturday, August 25, 2018, from 9:00 am - Noon. There will be a short interruption of service for each server during the maintenance window. Please forward any questions to rcc@partners.org.

There was a short interruption of service for the RISC Website during the maintenance window for MySQL Server patching on Saturday, August 25 between 9a.m.-Noon. The RISC Website, https://rc.partners.org, was unavailable while the system was rebooted during that window.

Please contact rcwebmaster@partners.org if you have any questions or concerns.

Applies To: All Users

Information Systems Maintenance (ISM) was conducted on Monday, August 20th from 12:00 am to 4:00 am.

During this time, planned system maintenance and testing was performed by IS Operations teams. This routine maintenance reduces the risk of unscheduled outages on our computer network and to the Internet.

Impact to systems and applications

• VPN should be available during the ISM.
• Partners eCare - unavailable from 1:00 am until 3:00 am
  • Impacts all PeC applications and related interfaces.
  • Impacts PeC dependent applications including Patient Gateway.
• HealthShare - unavailable from 1:15 am to 2:30 am
• Impacts all data interfaces and web services to PeC Production
• All Cache Classic applications, except web paging, will be down from 1:15 am to 2:30 am.
• Partners-wide: LMR, Medication and Allergy Services, CAS, EMPI
• BWH:  Historical Order and eMAR Viewer, other PeC dependencies
• MGH: MGH OR Application, Historical Order and eMAR Viewer, other PeC dependencies
• All Cache Classic and Ensemble data interfaces and web services will be unavailable from 1:15 am to 2:30 am.
• The Specific applications within Cache that will be down are listed below:

Impact to sites

• Services or resources hosted in the Partners HealthCare data centers may be unavailable during the ISM.
• Your site's IS Management team will provide more detailed communication about planned outages at your facility and will coordinate with specific groups that may be impacted.

Need help during the downtime? 

View the Broadcast Online. Please contact your IS Service Desk if you have any questions.

View the 2018 IS System Maintenance (ISM) Schedule in the IS Service Desk Knowledgebase (Partners login required).

There was a short interruption of service for the RISC Website during the maintenance window for MySQL Server patching on Saturday, July 28 between 9a.m.-10a.m. The RISC Website, https://rc.partners.org, was unavailable while the system was being rebooted during that window.

Please contact rcwebmaster@partners.org if you have any questions or concerns.

The ERIS Infrastructure team performed maintenance on mysql.dipr.partners.org, mysql2.dipr.partners.org, and mysql3.research.partners.org on Saturday, July 28, 2018, from 9:00 am - Noon. There was a short interruption of service for each server during the maintenance window. Please forward any questions to rcc@partners.org.

REDCap (https://redcap.partners.org/redcap) requires infrastructure updates completed Tuesday, July 24th at 7:45 AM - 8:45AM EDT. 

If you have any concerns about this scheduled maintenance, please email edcsupport@partners.org

There was a short interruption of service for the RISC Website during the maintenance window for MySQL Server patching that occurred on Saturday, June 23 between 9a.m.-Noon. The RISC Website, https://rc.partners.org, was unavailable for approximately five minutes while the system was rebooted during that window.

Please contact rcwebmaster@partners.org if you have any questions or concerns.

REDCap (https://redcap.partners.org/redcap) upgrades to v8.1.17 scheduled for Tuesday June 19th at 6:30AM EDT are completed. REDCap was offline from 6:30-7:34 AM EDT.

This upgrade included a number of new features and bug fixes. These include:

New action tag: @HIDECHOICE - Hides one or more choices of a multiple choice field. This action tag is useful if you wish to retire a particular choice after utilizing it for a while in data collection, thus allowing you to hide the choice from that point after without orphaning any of the choice's data, which would happen if you simply deleted the choice.

New feature: PDF download for survey respondents - On an instrument's survey settings page, a user may enable the option "Allow participants to download a PDF of their responses at end of survey?". This option will display a button for the survey participant to download a PDF file of their responses for the survey they just completed. Users may also download this same copy of the PDF since it has been added as a new PDF download option at the top of data entry forms.

Improvement: When a calc field's value is changed, it will now display a red border on the right edge of the calc field's text box to subtly indicate that the displayed value is different from the stored value in the database. And if the calculated value gets changed so that its value matches the stored value again, the red border will disappear. This will help users make note of when calculations have been triggered and if there is any disparity between the displayed value and the stored value.

New action tag: @MAXCHECKED - Allows a checkbox field to have a maximum number of checkboxes that can be checked.

New feature: New option to allow respondents to return to a partially completed or fully completed survey response without needing a return code.

List of more new features and fixes available here.

Full release notes available here.

If you have any concerns about REDCap being offline at this time or any questions about these new features, please email edcsupport@partners.org

We are sorry for any inconvenience that occurred with the proxy server outage and we want to share what happened and what we have done to remedy the situation. We were unaware of an increase in usage directed at our servers. All DIPR based HTTP and HTTPS traffic route through our proxy server in the Partners DMZ. We have seen an increase in traffic over the past few weeks and a very large increase in traffic on Wednesday, May 23rd at 9:00am, 2:30pm and 3:50pm.  This increase in traffic caused the proxy server to run out of memory. After the first incident, we increased the memory on the proxy server from 12gb to 16gb and engaged Neteng to try and determine where the increased traffic was coming from. After the third incident on Wednesday, a customer reached out to us concerning the stability of our environment and noted that they had increased their usage by 2500 new engagements to their site. Given this news we increased the memory on the proxy server from 16gb to 32gb, and we adjusted the apache configuration to manage the new load. We are currently monitoring apache, memory usage and load on the proxy server and feel that now we have a good configuration to manage the increased load. Also, we have started plans to replace the current proxy server with a redundant pair of proxy servers, however, that configuration is probably a couple of months away from implementation. 

Proxy users could help us by providing usage estimates for their sites and letting us know when they anticipate new users or additional sites and users coming online.

Research IS & Computing website maintenance completed on Thursday, May 24 between 7-7:30a.m. EDT. The https://rc.partners.org website was offline from approximately 7:10-7:15a.m. EDT to apply security patches and bug fixes.

This morning, there was an issue with our license in Production and our system was not able to send out any mailings, between 12:00 am to 10:40 am (this morning). The issue was corrected and the problem is now, fixed!

** If you sent or scheduled, any mailings between 12:00 to 10:40 am (today), please do not re-send the mailings, again. All mailings were queued and should be sending out as normal. If your mailings, do not get sent out in the next hour or so, please contact: rcc@partners.org for assistance in sending out your mailing.

Additionally, if you have further questions or concerns about this issue, please email: rcc@partners.org