Weekly

SiteReadiness: 100.0% SAM tests: 98.9% There was a problem with user CRAB jobs caused by CMSSW having problems with davs protocol. I switched our site to root default. Over last 24 hours CMS transferred to us ~600 TBs of data. Today we are scheduled for data challenge tests.

SiteReadiness: 100.0% SAM tests: 99.9% Overall uneventful week.

SiteReadiness: 100.0% SAM tests: 95.3% Overall uneventful week. The apptainer misconfiguration was discovered on several WNs, and corrected. We have somewhat low utilization of our site for another week.

SiteReadiness: 100.0% SAM tests: 99.9% Overall uneventful week. What worries me is low utilization of our site.

SiteReadiness: 85.7% SAM tests: 91.9% Over the last week we started to have problems with the worker nodes (WNs). Some WNs loose reliable connectivity to the outside world (ping starts showing packet loss), some go offline, and some need to be rebooted. I am not sure what is exactly the reason behind it – CMS … Read more

SiteReadiness: 100.0% SAM tests: 98.7% No significant issues over last week. I switched CRAB area (/store/temp) from HDFS to CephFS, preparing to switch /store/unmerged, and with that all CMS areas will be in CephFS.

SiteReadiness: 100.0% SAM tests: 99.0% No significant issues over last week. It looks like the storage is synchronized with Rucio. There were no FTS logs indicating that we miss some files or the files have incorrect checksum. Also no such new reports from users came. CephFS storage is sitting at ~74%, I am considering switching … Read more

SiteReadiness: 71.4% SAM tests: 100.0% The only issue over the last week, like in the previous week, was FTS errors. Together with DM team we took care of all files that were lost, and of all files that had incorrect check sum. I am monitoring FTS logs to catch cases when a transfer fails because … Read more

SiteReadiness: 71.4% SAM tests: 99.9% The only issue over the last week is FTS errors. Data Management team transferred all missing files. The remaining item was to take care of files that only existed at our site and were lost. They should have been invalidated globally, but instead DM team invalidated them only partially, as … Read more

SiteReadiness: 57.1% SAM tests: 90.3% I switched our site configuration on May 22. However, changes were propagated to /cvmfs only on May 27. For several days there was a difference between global and local site configuration. I am not sure if this is the reason behind our site in red for two days in site … Read more