# Monitoring Documentation

## List of all the alerts that are live

1. **Arb Block vs Subgraph Block**
Marlin-master : Subgraph is not in sync with blockchain
Subgraph link: https://thegraph.com/explorer/subgraphs/GUh83DEwZWMTkKaydusdkb46mLuAC7FTL4km1bcNjugc?view=Query&chain=arbitrum-one

2. **Loading Time Subgraph**
Marlin-master : Subgraph is taking too long to load
Subgraph link: https://thegraph.com/explorer/subgraphs/GUh83DEwZWMTkKaydusdkb46mLuAC7FTL4km1bcNjugc?view=Query&chain=arbitrum-one
3. **Loading time arb app website**
https://arb1.marlin.org website is taking too long to load
4. **Cluster info API**
https://sk.arb1.marlin.org/getClusterInfo API is down
5. **Contract details API**
https://sk.arb1.marlin.org/getContractDetails API is down
6. **Expected Reward API**
https://sk.arb1.marlin.org/getExpectedReward API is down
7. **Networks API**
https://sk.arb1.marlin.org/getNetworks API is down
8. **Oyster operator spec**
https://sk.arb1.marlin.org/operators/spec/ArbOne/0xe10fa12f580e660ecd593ea4119cebc90509d642 endpoint is down and not returning any data.
9. **Expected rewards**
https://sk.arb1.marlin.org/getExpectedReward is down and returning no data
10. **Cluster contract new transactions**
No new transaction made to https://arbiscan.io/address/0xf1e4c2d4b2996c7c34b5a6136861d7385cb0e843 in the last 30 minutes.
12. **Polygon operators info**
https://sk.arb1.marlin.org/operators/info/Polygon is down and not returning any data.
13. **Blog site is taking too long to load**
https://blog.marlin.org is taking too long to load
14. **Oyster indexer block number**
Marlin-master : Oyster indexer is returning the same block number for the last 10min.
https://indexer.oyster.marlin.org/graphql
15. **Serverless indexer block number**
Marlin-master : Serverless indexer is returning the same block number for the last 10min.
https://serverless-indexer.marlin.org/graphql
16. **Sheet hater bot server status**
Sheet hater bot server seems to be down : https://echo.sheethater.com/health
17. **KMS http endpoint**
KMS http endpoint is down ("http://image-v4.kms.box:1101/derive/x25519/public?image_id=bafc51da6492a2c316260eeab60b4249036b7664ad4617f5117826dbaa1d452e&path=gov_key")
19. **queued block**
queued block value has not changed in the last 2 hours for ETH proofs.
20. **RPC A**
https://arb1.arbitrum.io/rpc is not working
21. **RPC B**
https://arbitrum-sepolia.gateway.tenderly.co is not working
22. **RPC 3**
https://arbitrum.drpc.org is not working
23. **RPC 4**
https://mainnet.infura.io/v3/0672ea00967a4****7b418daf4a3
is not working
24. **Linea RPC status**
https://rpc.linea.build is down
27. **Arb One RPC status**
https://arb1.arbitrum.io/rpc is down
28. **Polygon RPC**
https://polygon-rpc.com/ is down
29. **SUI RPC**
https://rpc-mainnet.suiscan.xyz/ is down
30. **BSC RPC**
https://bsc-dataseed1.defibit.io is down
31. **ARB ONE CVM RPC**
https://1rpc.io/arb is down
32. **BSC CVM RPC**
https://bsc.drpc.org is down
33. **SUI CVM RPC**
https://fullnode.mainnet.sui.io:443 is down
34. **AMD64 PCR**
https://artifacts.marlin.org/oyster/eifs/base-blue_v3.0.0_linux_amd64.json is not returning PCR values in correct format
35. **ARM64 PCR**
https://artifacts.marlin.org/oyster/eifs/base-blue_v3.0.0_linux_arm64.json is not returning PCR values in correct format.
36. **amd64 CVM binary**
amd64 CVM binary is not downloadable.
37. **arm64 CVM binary**
arm64 CVM binary is not downloadable.
38. **darwin CVM binary**
darwin CVM binary is not downloadable.
39. **amd64 blue image**
amd64 blue image is not downloadable.
40. **arm64 blue image**
arm64 blue image is not downloadable.
41. **Oyster Simulate Docker**
Oyster simulate docker image is not available
42. **Pinata public gateway**
Pinata public gateway seems to be down - https://gateway.pinata.cloud/ipfs
43. **KMS contract root server**
KMS contract root server is not returning public address.
44. **ARB default operator**
ARB default operator seems to be down :
0xe10fa12f580e660ecd593ea4119cebc90509d642
45. **BSC default operator**
BSC default operator seems to be down :
0xD379eAb71eC709B20A85e953EbD3d395E032a9Af
46. **Operator Info endpoint**
Operator Info endpoint is down.
https://sk.arb1.marlin.org/operators/info/ArbOne/0xe10fa12f580e660ecd593ea4119cebc90509d642
47. **ARB reachability monitor**
(ArbOne) Multiple instances are failing the reachability test.
Dashboard for more error details : https://oyster-mp-dashboard.vercel.app
48. **ARB operator endpoint ip test**
(ArbOne) Operator endpoint is not returning a valid IP address for a job.
Dashboard for more error details : https://oyster-mp-dashboard.vercel.app
49. **SUI default operator**
SUI default operator seems to be down :
0x31777daafca0aac90f1171039735c87d3a933c95b997d34cc4fb2ea323f48ff4
50. **BSC reachability monitor**
(BSC) Multiple instances are failing the reachability test.
Dashboard for more error details : https://oyster-mp-dashboard.vercel.app/bsc
51. **BSC operator endpoint ip test**
(BSC) Operator endpoint is not returning a valid IP address for a job.
Dashboard for more error details : https://oyster-mp-dashboard.vercel.app/bsc
52. **SUI reachability monitor**
(SUI) Multiple instances are failing the reachability test.
Dashboard for more error details : https://oyster-mp-dashboard.vercel.app/sui
51. **SUI operator endpoint ip test**
(SUI) Operator endpoint is not returning a valid IP address for a job.
Dashboard for more error details : https://oyster-mp-dashboard.vercel.app/sui
52. **Kalypso-mainnet : Slash snapshot**
Kalypso-mainnet : Slash snapshot data not updated for the last 60hrs.
53. **Kalypso-mainnet : Vault snapshot**
Kalypso-mainnet : Vault snapshot data not updated for the last 60hrs.
54. **Kalypso-mainnet : latest block number**
Kalypso-mainnet : Latest block number has no value
55. **Kalypso-mainnet : ETH balance ( Address : 0x718B...a23e4 )**
Kalypso-mainnet : ETH balance below 0.01ETH (Address : 0x718BB6E41B33c26A563409e68B5dDe0b329a23e4)
56. **Kalypso-mainnet : Proofs generated has no value**
Kalypso-mainnet : Proofs generated has no value
57. **Kalypso-mainnet : Proofs generated**
Kalypso-mainnet : No Proofs Generatored in last 3 hours
58. **Kalypso-mainnet : Unhandled logs**
Kalypso-mainnet : unhandled logs has some new data.
59. **Kalypso-mainnet : Unmatched Jobs**
Kalypso-mainnet : There are more than 5 unmatched jobs
60. **Kalypso-mainnet : No new requests made**
Kalypso-mainnet : No new requests made in the last 3 hours.
61. **Kalypso-mainnet : Requests made has no value.**
Kalypso-mainnet : Requests made has no value
62. **attestation.ivs.nitro.kalypso.org status**
attestation.ivs.nitro.kalypso.org is down
63. **attestation.ivs.kalypso.org status**
attestation.ivs.kalypso.org is down
64. **verification.ivs.nitro.kalypso.org status**
verification.ivs.nitro.kalypso.org is down
65. **attestation.marlin.org status**
attestation.marlin.org is down
66. **Points endpoint status check**
Kalypso : https://points.kalypso.org/api/stats is down
67. **Count for kalypso points endpoint.**
Kalypso : https://points.kalypso.org/api/all is returning less than 50 objects.
68. **Kalypso points last block difference**
Kalypso : https://points.kalypso.org/api/last_block is returning outdated blocks.
69. **Arbitrum indexer lag status**
ARB indexer is laging behind the blockchain RPC by more than 100 blocks.

https://indexer.oyster.marlin.org/graphql
70. **Arbitrum serverless indexer lag status**
ARB indexer is laging behind the blockchain RPC by more than 100 blocks.

https://serverless-indexer.marlin.org/graphql
71. **BSC indexer lag status**
BSC indexer is laging behind the blockchain RPC by more than 100 blocks.

https://api.studio.thegraph.com/query/76614/marlin-oyster-bsc/version/latest

72. **SUI testnet indexer status**
SUI testnet indexer is down.

https://graph.lagoon.build/oyster/sui/graphql

73. **SUI testnet indexer lag status**
SUI testnet indexer is laging behind the blockchain RPC by more than 300 checkpoints.

https://graph.lagoon.build/oyster/sui/graphql

74. **BSC indexer status**
BSC indexer is down.

https://api.studio.thegraph.com/query/76614/marlin-oyster-bsc/version/latest

75. **Arbitrum indexer status**
ARB indexer is down.

https://indexer.oyster.marlin.org/graphql

## REPOS

Arb & BSC backend : https://github.com/marlinprotocol/evm-marketplace-monitoring-service

SUI backend : https://github.com/marlinprotocol/sui-marketplace-monitoring

Main monitoring service : https://github.com/marlinprotocol/monitoring