Research Computing Ticketing System Now Online
The email component of the internal ticketing system (Kayako) used by ERIS has not been working properly. The issues started Wednesday, May 29 around 12p.m. EDT and are not yet fully resolved.
The system may be delivering several of the same email or none at all. At times, ERIS was not receiving your tickets at all, so slightly longer than usual response times may be experienced.
Affected Services
Emails and requests sent to the following addresses were not accessed during this service disruption:
edcsupport@partners.org - REDCap-generated requests to approve, copy or move projects to production have not been received.
hpcsupport@partners.org
rcc@partners.org
ideasupport@partners.org
Please allow our staff time to catch up on the tickets submitted while the system was offline.
Microsoft has taken the unusual step of releasing sweeping system patches for older or unsupported but widely used Windows operating systems on Tuesday May 14 that include one critical patch that should be applied immediately. Details Here
The Oracle patches scheduled for the Partners Personalized Medicine team on Saturday morning is taking longer than anticipated and will extending past noontime. We anticipate another hour of patching and rebooting as we should complete our efforts by 1:00pm.
Please forward any questions or concerns to rcc@partners.org
The ERIS Infrastructure team will perform maintenance on the PostgreSQL Service on Saturday, April 27, from 9:00 a.m. - 12 p.m. EST.
Impact: All databases hosted on the ERIS PostgreSQL service will be impacted by one or two short reboots during this maintenance window.
Action Required: If you are the owner of a database on the PostgreSQL Service, we encourage you to check your applications after the maintenance window.
Completed
The ERIS Infrastructure team completed the on the MySQL hosted database Service on March 30, 2019. Please forward any questions to rcc@partners.org.
This website may have been offline for a brief period during the maintenance window for MySQL Server patching on Saturday, March 30 between 7:00a.m. - 10:00a.m. EDT. This website, https://rc.partners.org, may have been unavailable for a few minutes while the system is rebooted to apply patches.
Please contact rcwebmaster@partners.org if you have any questions or concerns
We are upgrading the List Manager (Lyris) system to v12.3.5 on Sunday, April 14, 2019, at 9 PM. We expect this release to take, up to an hour, so please expect the List Manager (Lyris) service to be offline from 9:00 to 10:00 PM. If you have any questions or concerns, relating to this upgrade, please email: rcc@partners.org.
Research Computing Ticketing System Online
The MySQL Hosted Database Services outage impacted the internal ticketing system (Kayako) used by ERIS. The outage started Thursday, March 21 around 10p.m. EDT and services restored around 12:15p.m. EDT on Friday, March 22.
Affected Services
Emails and requests sent to the following addresses were not accessed during this unanticipated downtime:
edcsupport@partners.org - REDCap-generated requests to approve, copy or move projects to production have not been received.
hpcsupport@partners.org
rcc@partners.org
ideasupport@partners.org
Please allow our staff time to catch up on the tickets submitted while the system was offline.
RC Services Online
The outage started Thursday, March 21 and we expect services to be restored around 12:30p.m. EDT on Friday, March 22. This downtime is related to the MySQL Hosted Database Services outage.
We will update this message when service is restored.
MySQL Hosted Database Services Online
The outage started Thursday, March 21 around 10p.m. EDT and was restored around 12:00p.m. EDT on Friday, March 22.
Action Required
If you are the owner of a system on mysql.dipr.partners.org, we strongly encourage you to check your application connection and reboot if necessary.
Affected Services
All systems hosted on mysql.dipr.partners.org
Research Computing ticketing system (Kayako tickets.partners.org)
Research Computing ordering/request system (RC Services rcservices.partners.org)
We will update this posting once service has been restored. Please contact rcc@partners.org with any questions or concerns.
REDCap (https://redcap.partners.org/redcap) upgrades to v8.5.28 scheduled for Tuesday March 19th at 6:30AM EDT are complete REDCap was offline from 6:30AM-6:48AM EDT.
This release includes a number of bug fixes.
If you have any concerns about REDCap being offline at this time, please email edcsupport@partners.org
The hpcwin3.partners.org Windows analysis server has presented several freeze instances in the last few days. To evaluate the usage and logs we need to take the server out of service for a few days. During this window time we will perform test and review the information of the processes that have been causing the system to crash. Future actions needed to be taken after for repair or control will be announced to the users.
Existing remote desktop sessions will be terminated. Please save all open files and exit from your remote desktop session as soon as possible. We apologize for the inconvenience this may cause.
We had to reboot the service. This is an NFS server serving mapped drives fro /nfs1, /nfs2, and /redcap-prod. The fs.dipr.partners.org VM is currently online, however, you may need to remount your NFS mounts this morning.
REDCap (https://redcap.partners.org/redcap) requires server maintenance which is scheduled for Saturday, February 23rd 2019 from 7:00AM to 9:00AM EST. There will be a short interruption of service during the maintenance window.
If you have any concerns about this scheduled maintenance, please email edcsupport@partners.org
There was a short interruption of service for the RISC Website during the maintenance window for MySQL Server patching on Saturday, February 23 between 7:00a.m. - 9:00a.m. The RISC Website, https://rc.partners.org, was unavailable for a few minutes while the system is rebooted during that window.
Please contact rcwebmaster@partners.org if you have any questions or concerns
The ERIS Infrastructure team will perform maintenance on mysql3.research.partners.org and mysql2.dipr.partners.org on Saturday morning February 23, 2019, from 7:00 am - 9:00 am. There will be a short interruption of service during the maintenance window. All databases hosted on mysql3.research.partners.org or mysql2.dipr.partners.org will be offline while we shut down and restart each node.
Action Required
If you are the owner of a database on mysql3.research.partners.org or mysql2.dipr.partners.org we encourage you to check your applications after the maintenance window.
The GitLab service (https://gitlab.partners.org/) will be upgraded from version 11.5.7 to version 11.7.5 and the upgrade is scheduled to take up to 1 hour. During this time, the service will be unavailable between 7 AM to 8 AM on Monday, February 25th. This upgrade will include: New features, critical security patches, and bug fixes
For a complete list of changes, please refer to the GitLab project release notes:
*** This is a major version upgrade of GitLab. As a result, the underlying tools, frameworks, and programming languages that GitLab is built upon have also been updated, most notably the Ruby Rails version. This should only affect functionality in the GitLab user interface. The GitLab API version has not been updated - remains at v4.
We recommend that you test your processes (in the development system, at this link: http://gitlab-dev.dipr.partners.org - to log in, just use your Partners ID and Password). The Development server is currently at version 11.7.10. No projects or data have been copied over from the production server. You can create new projects in the development environment and conduct tests as necessary. Projects created in the development environment may be removed without notice (as we conduct future GitLab version testing). Please complete any necessary testing before the upgrade on Monday, February 25th at 7 AM.
If you have any other questions or concerns, relating to this scheduled downtime, please email: rcc@partners.org
Partners HealthCare GitLab: Scheduled Downtime on 2/04/2019
The GitLab service (https://gitlab.partners.org/) will be upgraded from version 11.5.6 to version 11.5.7. The upgrade is scheduled to take up to 1 hour. During this time, the service will be unavailable between 7 AM to 8 AM on Monday, February 4th.
This upgrade will include:
For a complete list of changes, please refer to the GitLab project release notes:
** Additionally, we want to reach out to the users of GitLab, to complete any necessary testing before the upgrade on Monday, February 4th at 7 AM. Please complete all your testing at this link: http://gitlab-dev.dipr.partners.org (** To login, just use your Partners ID and Password). The Development server is currently at version 11.5.7. Note – no projects or data have been copied over from the production server. You can create new projects in the development environment and conduct tests as necessary. Projects created in the development environment may be removed without notice (as we conduct future GitLab version testing).
If you any questions or concerns relating to this scheduled downtime, please email: rcc@partners.org
