Service Alerts

yellow - pre-downtime, calls to action

Simches Datacenter UPS Maintenance 10 Dec 2019 6AM EDT

December 10, 2019 6:00 am to 8:00 am

Annual service and preventive maintenance of the UPS and Backup batteries will be performed next week. The UPS will be placed into bypass on utility power only. 

Service Date/Time
Date: Tuesday, December 10
Time: 6 AM to 8 AM EDT
ServiceNow Change #: CHG0157000

Please contact rcc@partners.org if there are any questions or concerns!

green - services online

List Manager Limited Support

November 21, 2019 8:00 am to December 2, 2019 8:00 am

From Thursday, November 21, 2019 through Monday, December 2, 2019 at 8a.m., there will be limited List Manager (Lyris) support. The infrastructure will be supported, so if there are concerns with connection to https://researchlistadmin.partners.org or mail delivery, they will be addressed and resolved in the normal response time.

Requests for new lists and emails regarding additional support will be responded to starting December 2, 2019.

For answers to commonly-asked questions and requests, please refer to the following resources:

green - services online

REDCap Restored 11 Nov 2019

November 10, 2019 9:00 pm to November 11, 2019 2:00 pm

REDCap (https://redcap.partners.org/redcap) was currently experiencing sporadic WebDAV/file server issues. These include users not being able to export data, upload files, or access uploaded files. 

If you have any concerns, please email edcsupport@partners.org

This is related to the research website and service outages that started on Sunday, November 10, 2019 around 9p.m. 

The root of the problem is fixed and was related to scheduled storage and infrastructure maintenance. However, as of Monday, November 11, 2019 at 11a.m. EDT, not all services are back online.

If you are the owner of websites, hosted databases, storage or virtual servers on ERIS infrastructure, we encourage you to check your connection and reboot if needed.

green - services online

GitLab Restored 11 Nov 2019 10:30PM EDT

November 10, 2019 9:00 pm to November 11, 2019 10:30 pm

The GitLab service (https://gitlab.partners.org/) has been restored after a major outage.

A major storage outage incident took place during a scheduled storage controller upgrade task. The incident affected GitLab as well as many other systems. The GitLab system was unavailable from approximately 9:50PM on Nov 10th until 10:50PM on Nov 11th. Due to the large amount of data in the GitLab repositories and other connected storage systems, the disk check and repair operation took much longer than expected, which caused the extended outage. The issue was subsequently compounded by a networking-related reconfiguration (on the storage-side).

At this point we do not believe that any data was affected. Please double-check your repositories, though!

We appreciate your continued usage and support of the service! We are working to make improvements of the service to avoid instabilities such as this one in the future.

If you any questions or concerns relating to this release, please email: rcc@partners.org.

green - services online

Hosted Services Restored 11 Nov 2019

November 10, 2019 9:00 pm to November 11, 2019 3:30 pm

Several research websites and services were unavailable from Sunday, November 10, 2019 starting around 9p.m. until Monday, November 11 around 3:30p.m. EDT.

The root of the problem was related to scheduled storage and infrastructure maintenance.

If you are the owner of websites, hosted databases, storage or virtual servers on ERIS infrastructure, we encourage you to check your connection and reboot if needed.

Contact rcc@partners.org to report any remaining issues or for assistance. 

green - services online

ERIS Website Hosting Restored 11 Nov 2019

November 10, 2019 9:00 pm to November 11, 2019 12:30 pm

ERIS Website Hosting Services are restored as of 12:30 PM EDT on Monday, November 11, 2019. Several research websites and services were unavailable beginning on Sunday, November 10, 2019 starting around 9PM.

The root of the problem is fixed and was related to scheduled storage and infrastructure maintenance

If you are the owner of a website, hosted database, storage or virtual server on ERIS infrastructure, we encourage you to check your connection and reboot if needed.

Contact the IS Service Desk to report any remaining issues or for assistance.
 

green - services online

REDCap Scheduled Downtime

November 1, 2019 10:12 am

REDCap (https://redcap.partners.org/redcap) server maintenance is scheduled for Saturday, November 2nd, 2019 from 9:00AM to 2:00PM EST. Please expect REDCap to be down for a short time during the maintenance window.

If you have any concerns about this scheduled maintenance, please email edcsupport@partners.org

yellow - pre-downtime, calls to action

Compellent Controller Upgrade

November 10, 2019 2:00 pm

On Sunday, November 10, 2019, Dell Compellent SAN Controllers were replaced with new SC9000s. Many of our services rely on Compellent Storage Area Network (SAN) for underlying storage. This maintenance impacted connectivity to the storage (RFA, RFANFS, DIPR, etc.) being served out by this system.

As of Monday, November 11, 2019 at 9a.m. EDT, not all services are back online.

If you are the owner of a website, hosted database, storage or virtual server on ERIS infrastructure, we encourage you to check your connection and reboot if needed. Contact the IS Service Desk to report any remaining issues or for assistance.

Please check the Service Alerts for service-specific instructions and impact. Please make sure you subscribe to receive appropriate email notifications for the services you subscribe to: https://rc.partners.org/subscribe.

green - services online

RISC Website Maintenance 28 Sept 2019

September 28, 2019 9:00 am to 2:00 pm

The https://rc.partners.org website was offline for a short period during routine maintenance on Saturday, September 28 from 9a.m.-2p.m. EDT.

Please contact rcwebmaster@partners.org with any questions or concerns.

green - services online

RISC Website Maintenance 22 Sept 2019

September 22, 2019 9:30 pm to September 23, 2019 12:00 am

The https://rc.partners.org website was offline for a short period of time for infrastructure maintenance that took place Sunday, September 22 from 9:30p.m. to midnight EDT.

Please contact rcwebmaster@partners.org with any questions or concerns.

green - services online

Completed MySQL PostgreSQL and Storage Maintenance

September 22, 2019 9:30 pm to September 23, 2019 12:00 am

The MySQL PostgreSQL and Storage Maintenance on Sunday Evening has Completed. 

The MySQL, PostgreSQL, and Storage Maintenance that was scheduled for Saturday, September, 21st 2019 at 12:00 pm have been rescheduled to Sunday evening September 22nd, 2019 at 9:30 pm. The ERIS Infrastructure team will shutdown all hosted PostgreSQL and MySQL servers at 9:45 pm on Sunday, September 22nd, 2019. The window for this maintenance is from 9:30 pm - 12:00 am. All databases hosted on mysql.dipr.partners.org, mysql2.dipr.partners.org, mysql4.research.partners.org, pgsql.dipr.partners.org, and pgsql2.research.partners.org will be offline during this maintenance window.

If you are the owner of a database on the PostgreSQL or MySQL hosted database service, we encourage you to review and validate your application's connection to your database when this maintenance has completed. 

yellow - pre-downtime, calls to action

MobileIron and iOS 13 and 13.1

September 24, 2019 1:00 pm

Applies to: iOS Users enrolled in MobileIron

The Partners MobileIron/EMM team has completed testing on latest GM build of iOS 13.1 Beta. Apple released iOS 13.0 on Thursday, 9/19 at around 1pm EDT and will release 13.1 on Tuesday, 9/24 around 1p.m.

  • If you upgrade to iOS 13.1 from iOS 13.0, no action is required
  • If you upgrade to iOS 13.1 from iOS 12, please note that you will be required to re-enter your Partners Email Password. A pop up will automatically appear that will take the you to the Settings App to re-enter your password.

If you accidentally hit “Cancel”, you can go to Settings > Passwords & Accounts > PARTNERS (OAUTH) to re-enter your Partners password.

Updated MobileIron enrollment instructions have been posted to the Knowledge Base 

 

IS Service Desk https://www.partners.org/ISServiceDesk 

BWH 617-732-5927      BWH-RICS 617-525-0848      CDHC 413-582-5005      DFCI 617-632-3399         
FALK 617-983-7454     LCC 857-307-4150                  MCL 781-416-8940        MGH 617-726-5085     
NHP 617-643-2020       NSMC 978-354-2014               NWH 617-243-6001       PCPO 857-282-5999     
PHH 617-726-0790       PHS 857-282-4357                   SRN 617-952-5555  
 

yellow - pre-downtime, calls to action

Windows 10 Upgrade for Non-standard PCs

January 6, 2019 3:54 pm

Microsoft is ending support for Windows 7. If you are currently running Windows 7 or older, you must upgrade to Windows 10 Professional by Jan. 1, 2020.

Vulnerabilities are discovered frequently in software and they put your computer, data and the enterprise at risk. In order to stay secure and compliant with Partners policies, make sure that you are running a current, supported Operating System (OS) and that your software applications are up to date. 

  • Vendor-provided device:  Work with the vendor. If the vendor device is not compatible with Windows 10.  An approved variance from Information Securityrequired.  
  • Partners Standard Windows devices: Devices are being upgraded by your IS Department. 
  • Non-Standard Windows PC: Follow the instructions below.

​HOWTO: Determine whether you are using a standard or non-standard computer 

Check the eligibility of your PC
Please review the Windows 10 minimum requirements. Check which edition you're using by right-clicking the Start button, clicking System, and finding the Windows Edition.

Which scenario best describes your non-standard Windows PC?
The device runs Windows 7, 8 or 10 Home Edition.

Option 1
Contact your purchasing department to acquire best pricing for Windows 10 Professional upgrade software.

Option 2
Replace device. Contact the IS Service Desk to have a ticket opened and sent to the ERIS site team to discussion options and compatibility.
The device runs Windows 7 or 8 Pro Edition.

Option 1
If you are already running a valid copy of Win7 Pro you can use the link below to download Win10 Pro and perform the upgrade. https://www.microsoft.com/en-gb/software-download/windows10ISO

Option 2
Contact your purchasing department to acquire best pricing for Windows 10 Professional upgrade software.

Contact the IS Service Desk with any questions or for technical assistance.

green - services online

DIPR Hosted Servers OS Patching 28 Sept 2019

September 28, 2019 9:00 am to 2:00 pm

Applies to: All DIPR Servers

Production servers hosted by ERIS were patched on Saturday, September 28 starting at 9AM. If you are the owner of a server hosted by ERIS, we encourage you to connect to your server at the end of our maintenance, verify that your applications restart and reconnect to your database. 

The Discovery Informatics Platform for Research (DIPR) platform hosted by Enterprise Research IS is undergoing several essential upgrades in Fall 2019 to elevate security standards. These changes will include increased auditing, enabling SELinux on all systems and patching of the Linux OS on all servers on a monthly basis. The ERIS Infrastructure team will communicate each change two weeks before deploying. Please subscribe to receive DIPR communications and visit https://rc.partners.org/subscribe for more information. Non-production DIPR servers were patched in waves on Wednesdays in September. 

We understand that this is a significant change. We need your help to ensure an easy transition for your servers, applications and users. Learn more about the new DIPR Servers Monthly OS Patching Schedule.

Service Level Reminders
To ensure premium supportability, security and performance, ERIS strives to keep all systems up to their most current and/or stable revisions. Please visit https://rc.partners.org/sla to review the full ERIS Service Level Agreement.  

Server owners are responsible for maintaining the applications running on DIPR VMs systems and ensuring they are patched and comply with Partners Information Security Practices and Policies. Server owners are responsible for backup of critical configurations in case of a failure.

Please reach out to rcc@partners.org with any questions.

green - services online

Complete ERIS MySQL and PGSQL Maintenance

August 24, 2019 10:00 am to 11:00 am

The ERIS Infrastructure team completed the maintenance on the ERIS MySQL and ERIS PGSQL servers. All servers and databases are up and online. If you are the owners of a ERIS Hosted Database, please check your applications connection to the database. Please contact rcc@partners.org with any questions.

green - services online

Research Computing Website Downtime 24 Aug 2019

August 24, 2019 9:00 am to 12:00 pm

This website was offline for a brief period during the maintenance window for MySQL Server patching on Saturday, August 24 between 9:00a.m. - Noon. EDT. This website, https://rc.partners.org, was unavailable for a few minutes while the system is rebooted to apply patches.

Please contact rcwebmaster@partners.org if you have any questions or concerns.

green - services online

REDCap Scheduled Downtime: August 24th, 2019 9:00-10:00AM EST

August 24, 2019 9:00 am to 10:00 am

REDCap (https://redcap.partners.org/redcap) server maintenance is scheduled for Saturday, August 24th 2019 from 9:00AM to 10:00AM EST. There will be a short interruption of service during the maintenance window.

If you have any concerns about this scheduled maintenance, please email edcsupport@partners.org

yellow - pre-downtime, calls to action

FreezerWorks: Expected Downtime 8/18/2019

August 18, 2019 1:00 am to 5:00 am

On August 18th from approximately 1:00 am to 5:00 am, Freezerworks will temporarily be offline to complete planned server maintenance work. During this timeframe, we ask users to log out of the system so we can complete this necessary maintenance work. We do apologize for any inconvenience this interruption may cause. 

If you have any questions or concerns, please email edcsupport@partners.org

yellow - pre-downtime, calls to action

List Manager (Lyris): Downtime on 18 August 2019

August 18, 2019 8:00 pm to 8:30 pm

We are upgrading the List Manager (Lyris) system to v12.3.6 on Sunday, August 18, 2019, at 8 PM. We expect this release to take, up to 30 minutes, so please expect the List Manager (Lyris) service to be offline from 8:00 PM to 8:30 PM

Applies to:
https://researchlistadmin.partners.org 
https://researchlist.partners.org

This upgrade was to improve system performance and address defects. The release notes can be found in the Knowledge Base

Please contact rcc@partners.org with any questions, concerns or issues with the upgrade.

yellow - pre-downtime, calls to action

Microsoft Windows Patch Tuesday

August 13, 2019 9:00 am

Did you know that Microsoft regularly releases security patches for its software products on the second Tuesday of every month? How frequently do you update your computer?

This month, Microsoft introduces patches for critical flaws and vulnerabilities that require you to patch as soon as possible.

Here is some guidance to keep your device secure and your Microsoft software up to date and running smoothly.

Do I need to make changes on my computer?

  • Not if you use a Partners standard Windows computer (those with the asset tag, yellow lock and Partners' screensaver). The updates are run for you and devices are usually patched the Friday following Patch Tuesday. Both Microsoft and Adobe products are typically updated at this time. This can change if there is a significant freeze in place. 
  • Yes, you need to run updates if you use a non-standard Windows, Macintosh or personally-owned device for Partners' business. ​It is your responsibility to keep your device and software up to date. 

See HOWTO: Run Windows Updates for additional instructions.

green - services online

ERIS Hosted Services

August 6, 2019 10:30 am
Several Enterprise Research IS services are offline starting at 10:30a.m. EDT on Tuesday, August 6, 2019. There was an issue with the underlying storage infrastructure. 

If you find a Virtual Machine (VM) or website that is not responding, please submit a ticket to the IS Service Desk

Additional Impacted Services:

Service Description Status Notes 
Impacted Services
ERIS MySQL hosted database services
  • Online as of 1:00p.m. EDT on 8/6/19 (server)
Research Computing Website (https://rc.partners.org

Online as of 2:49p.m. EDT on 8/6/19

ERIS Web Hosting Services Online as of 3:47p.m. EDT on 8/6/19
RFANFS
Online as of 2:17p.m. EDT on 8/6/19, Rebooting this system on 8/7/19 at 9p.m. EDT
RFAWIN
Online as of 7:00p.m. EDT on 8/6/19
REDCap (https://redcap.partners.org)
 
  • Online as of 1:47p.m. EDT on 8/6/19
  • Issues resolved as of 4:30p.m. EDT on 8/6/19. If you are still experiencing any issues related to logging into REDCap or importing/exporting data, please email edcsupport@partners.org.
Confluence/JIRA
Online as of 1:00p.m. EDT on 8/6/19
PEAS (https://enroll.partners.org)
Online as of 1:00p.m. EDT on 8/6/19
GitLab (https://gitlab.partners.org)
Online as of 1:00p.m. EDT on 8/6/19
RC Services (https://rcservices.partners.org)
Online as of 5:00p.m. EDT on 8/6/19
DIPR Virtual Machines (VMs)
Online as of 9:45p.m. EDT on 8/7/19
DIPR File Server (fs.dipr)
Online as of 9:45p.m. EDT on 8/7/19

If you find a Virtual Machine (VM) or website that is not responding, please submit a ticket to the IS Service Desk

green - services online

Securing your Network Connection at Assembly Row

August 13, 2019 7:00 am

Applies to:        Staff and Visitors at Assembly Row

The Network Access Control (NAC) program updates scheduled for Assembly Row on Tuesday, August 13 are canceled at this time. 

Partners HealthCare continues to enhance the Network Access Control (NAC) program at Assembly Row, strengthening visibility and security of data and devices connected to the Partners network.  

Please review the security requirements and guidance for your device, by visiting rc.partners.org/computer

Thank you in advance.

Jigar Kadakia
Partners Chief Information Security & Privacy Officer
CISPO@partners.org

yellow - pre-downtime, calls to action

Install CrowdStrike on Your Apple Macintosh Device

July 25, 2019 9:00 am

Beginning July 25, 2019, CrowdStrike will be pushed to your corporate/Partners-purchased Macintosh device. Please note the date was moved one week due to the Enterprise Change Freeze.

It will be required to connect to the Partners Corporate network. CrowdStrike is an additional endpoint security tool application that complements antivirus (AV). Both CrowdStrike and antivirus are required to keep your systems safe and secure. Trend Micro antivirus is available for installation on all Mac and Windows devices used for Partners business purposes, including research. Please see Supported Antivirus Software for a full list.

CrowdStrike end point protection and Trend Micro antivirus are available for download from the Partners Enterprise Apple Support (PEAS) Self Service application. 

CrowdStrike can also be self-installed prior to the automatic push using the directions below.

Please note: Apple released the public beta version for the next operating system for Mac desktops & laptops, macOS Catalina (10.15). We recommend you do not upgrade at this time. Traditionally new Mac Operating Systems will initially cause compatibility issues with Partners services.

To download CrowdStrike via Self Service:

  1. Launch “Self Service”
  2. In “Applications”, search for “CrowdStrike Falcon Sensor”
  3. Select Install

View detailed installation instructions on our Knowledge Base
 
For assistance or questions, contact the IS Service Desk and ask to submit a ticket to the Engineering Security Team.
 
Thank you in advance.
 Jigar Kadakia
 
Partners Chief Information Security & Privacy Officer
399 Revolution Dr, Somerville, MA 02145

green - services online

Mysql3 Database Move to Mysql4

July 19, 2019 6:01 pm to July 21, 2019 6:01 pm

This is an update on the ERIS Infrastructure teams efforts to move all databases off of mysql3.research.partners.org and restore them on mysql4.research.partners.org. The team has worked through 50% of the efforts needed to complete this effort and is working diligently on all remaining tasks. We will share another update later tonight and will provide more detail concerning availability.

The final step in this process belongs to the database owners who need to change their MySQL Data Source reference from mysql3.research.partners.org to mysql4.research.partners.org.

Please send a note to rcc@partners.org with any questions.

green - services online

Partners HealthCare REDCap: Scheduled Downtime

July 17, 2019 2:05 pm

IMPORTANT NOTICE: REDCap Database Migration

In order to meet the increased demand, usage, and amount of data stored within the REDCap system, it is necessary to upgrade the database infrastructure.

As a result, a REDCap database migration is scheduled for Saturday July 20th, 2019 at 7:00PM EST.  This will require an extended amount of downtime with REDCap being offline for approximately 14 hours. 

REDCap was offline from 7:00pm-5:55am. 

This will help optimize our REDCap infrastructure, resulting in faster overall performance, and prevent extended downtime for maintenance in the future. 

If you have any concerns about REDCap being offline at either of these times, please email edcsupport@partners.org

Pages