# Harmony Infinite Minting Bug Report (Emergency Patch - Dec 17, 2023 GMT-8) By Aaron Li ([@polymorpher](https://twitter.com/polymorpher)) (hiddenstate.xyz / modulo.so / … ) on Dec 16, 2023, 12pm (unaffiliated with Harmony) Current revision: Jan 18, 2024 ## Summary About 79 accounts (including some double counts) undelegated from 4 validators are credited with erroneously minted ONEs, resulting in a total of ~146.28M extra ONEs minted as of epoch 1732, peaking at ~9.09M extra ONEs minted every epoch to these accounts. An emergency patch was released for epoch 1733 (around ~4:30 am Dec 17, 2023 US Pacific Time) to stop the extra minting. However, as of now most of the existing extra ONEs are already sold or transferred and there is little we can do about them. Use my tool to check: [github.com/polymorpher/harmony-mint-bug-scan](https://github.com/polymorpher/harmony-mint-bug-scan). Luckily, ~51.2M of the extra ONEs (~$768k USD) are minted to my account, averaging around $90k - $100k USD per day based on market price. For full disclosure, I [sold 16.4M](https://docs.google.com/spreadsheets/d/13EoFAkKBFh97abG6U7rkjDA5RHipsCXNZ_zfhz3X1gs/edit#gid=0) ONE at around these times, which is less than what I legitimately owned on Harmony before the bug. The extra ONEs are a fortune, but I believe it belongs to the people in the sense that such extra minting affects all ONE holders. I am not taking the erroneously minted ONEs for personal gain, and I agree that these extra ONEs should be disposed of in a way that benefits the project the best, such as to burn, or to be used for public good (recovery, victim relief, defi, …). Casey Gardiner (Harmony protocol engineer) strongly believes they should be burned and has gone through many extreme actions to compel me to do so, some of which went beyond the line of civility and reasonableness. While I respect his zeal and enthusiasm, I don’t believe it is the right approach. Even if burning is the right choice, I believe we should first take input from the validators or community before taking actions, and we must explain how we reach the decision. The project’s success and prosperity are important to me. This report provides a full disclosure of how the bug was discovered, communicated, tested, fixed, and deployed, how the decisions were made, and my roles in all these. Drama, arguments, and clashes of personalities were inevitable in a crisis like this, and people say regrettable things under stress, with heightened emotions. In my report, I would like to focus on the factual data points, and minimize negativity, personal vendetta, and regrettable communications between parties. While burning is a natural course of action and favored by many, there are concerns that it might not help much compared to the ~146.28M extra ONE minted, and alternatives should be considered. After consulting with Recovery custodian validators, partners, and several independent validators, it is agreed upon that **burning** the extra ONEs is the best course of action. The primary reason behind this decision is that, even though using the funds for recovery or defi may provide users some temporary relief, the uncertainty of circulating supply and inflation may raise widespread concern and create far more harm to the project and to everyone. As such, I have commenced the initial [burning of 26.4M ONE](https://explorer.harmony.one/tx/0xb529924711e089836c16e09761aabf34762069989bf6d256f0880ce784984127) and [12.8M ONE](https://explorer.harmony.one/tx/0x6b21695d0a9f71a0d4877a18aa94c9add5fe99f2464d70232d9f4bc27e8e322b) and [3.4M ONE](https://explorer.harmony.one/tx/0xffa078bbcf0f9b81a6a966bf74f41acac42320e306005a9af2257f099de34afb), which will soon be followed with the remaining amount within the next 7 days, pending the team to fix some malfunctioning whitelist and blacklist features. ## Technical Overview To summarize the technical issue, some old code (~2020) [failed to check errors while persisting validator state updates](https://github.com/harmony-one/harmony/blob/ae578ba9983811d69b1ad228069b45f82c3628ed/core/state/statedb.go#L897) at an epoch boundary. Some of these errors would result in validator information staying stalled instead of properly persisted in the state database, such as which undelegations are yet to be matured and included in the payout. Since [payouts are made regardless](https://github.com/harmony-one/harmony/blob/ae578ba9983811d69b1ad228069b45f82c3628ed/internal/chain/engine.go#L281C22-L281C22) of whether the validator state update is successful, this results in the delegators repeatedly getting paid at every epoch for whatever undelegations they made at least 7 epochs ago. The exact error triggered during validator state update is a [sanity check](https://github.com/harmony-one/harmony/blob/ae578ba9983811d69b1ad228069b45f82c3628ed/staking/types/validator.go#L315) that fails for some validators after the most recent HIP30 updates that raised minimum commission from 5% to 7%. These validators, for historical reasons, have both a commission rate set at 5%, and an internal “max commission rate” set at 5%. While the recent HIP30 update [programmatically adjusts](https://github.com/harmony-one/harmony/blob/ae578ba9983811d69b1ad228069b45f82c3628ed/internal/chain/engine.go#L290C3-L290C3) all validators’ current commission rate to at least 7%, it overlooks the “max commission rate” part. Hence, at the end of every epoch, after the program tries to raise these validators’ commission rate to 7%, the aforementioned sanity check failure gets triggered because the raised commission rate (7%) is now greater than the internal “max commission rate” (5%). Consequently, the validator state update fails silently while payout for undelegations is already made, whereas the matured undelegations that are already paid out is not removed from the validator’s state. This scenario would repeat over and over for every epoch, and more undelegations may be added to the payout list when the delegators undelegate, resulting in possibly an infinite amount of ONE get minted if someone deliberately and maliciously undelegate and delegate over and over at every epoch. i.e. it could blow up exponentially. If one starts with 1M ONE staked at epoch T, they could undelegate 1M at epoch T, and re-delegate the 1M again at epoch T+1, then undelegate at T+1, re-delegate at T+2, and repeat. By epoch T+7, they would receive 1M from bad undelegation. At T+8, they would receive another 3M (= 1M + 2M). At T+9, they would receive another 6M. At T+10, they would receive another 10M, and so on. Note that each time they receive something, they could immediately delegate them and undelegate just like what they did at epoch T, only at a much bigger scale each time. Very luckily, I caught the bug before anyone started to maliciously exploit it, and before anyone received a large amount of payout. The destruction of Harmony and everyone’s assets would be certain, had it been the case. Based on the result of my scan tool, only four validators were affected ([H1](https://github.com/polymorpher/harmony-mint-bug-scan/blob/30ab393f262a42f09a75cbcfcd77e39a055e66a8/don.txt), [Tr4ck3r](https://github.com/polymorpher/harmony-mint-bug-scan/blob/30ab393f262a42f09a75cbcfcd77e39a055e66a8/tr4.txt), [Husaria](https://github.com/polymorpher/harmony-mint-bug-scan/blob/30ab393f262a42f09a75cbcfcd77e39a055e66a8/husa.txt), and [Sargatxet](https://github.com/polymorpher/harmony-mint-bug-scan/blob/ccb4a00acb7ea7c01df148f3e219c1445e4b5fba/sargatxet.txt)), and the data suggests all delegators who received the extra payout were unintentional. Based on explorer data, many of them were unaware that they got paid extra, though a few individuals who received a good amount of money became aware of the issue and had been continuously selling whatever they received as quickly as possible. And they did not report the issue to anyone. ## Timeline 1. On Dec 7, 2023 at 2:35pm, I discovered irregularity when my un-delegated ONEs were returned to me. I immediately brought the issue to the attention of Casey Gardiner, Li Jiang, and Stephen Tse. See screenshot below. This particular transaction is linked to my wallet [0x18a5af69dfc02dc950f7534ae97a180f34b75d7a](https://explorer.harmony.one/address/0x18a5af69dfc02dc950f7534ae97a180f34b75d7a?activeTab=0), an address that is on numerous committee and their multisig wallets, such as the transaction fees committee and modulo recovery multisig. I received a reaction from Stephen but not from Casey, who is responsible for looking into protocol issues such as this. [Screenshot available upon request](https://drive.google.com/file/d/1fusUgFFIrGXy84r2IPtBmqHMz4YvzxPH/view?usp=drive_link) > [Group chat: Aaron Li, Li Jiang, Casey Gardiner, Stephen Tse] > > (December 7, 2:23pm) > > **Aaron**: https://explorer.harmony.one/tx/0x818a3e96f03d32778165fb8ac92cb8651d3291fc9d1d0fe4e0302c7633a8a435 an undelegate of 1.4M somehow resulted the wallet having 2.8M when it is done > > ![2023-12-07-disclosure-attached-1](https://storage.googleapis.com/harmony-infinite-minting-files/2023-12-07-disclosure-attached-1.jpg) > > ![2023-12-07-disclosure-attached-2](https://storage.googleapis.com/harmony-infinite-minting-files/2023-12-07-disclosure-attached-2.jpg) > >![2023-12-07-disclosure-attached-3](https://storage.googleapis.com/harmony-infinite-minting-files/2023-12-07-disclosure-attached-3.jpg) > > (Reaction from Stephen: 😢) 2. On Dec 8, 2023 at 11:37am, I followed up on the issue again and pinged Casey. I received a response from Stephen, but no response and no action was taken by Casey. > [Group chat: Aaron Li, Li Jiang, Casey Gardiner, Stephen Tse] > > (December 8, 11:37am) > > **Aaron**: I think it is very likely there is some problem here. Last night the address had 1M, and pending 3M being undelegated to be available at the end of the epoch. Now it has 5.4M. So 1.4M came out of nowhere again *@caseyga* > > (Reaction from Stephen: 😱) 3. I find it both strange and scary that such an egregious issue is ignored. I sold 14.6M ONEs across Dec 8 - 10, 2023 using the same wallet. I deliberately controlled the amount to be less than the number of ONEs I legitimately own, had the bug not occurred (for example, I also own [0x14b6e8df446e81c7f2c69ba1573197ccebab1d61](https://explorer.harmony.one/address/0x14b6e8df446e81c7f2c69ba1573197ccebab1d61) which has 10M+ ONEs [pending undelegation](https://explorer.harmony.one/tx/0xbb734439227867e53dd41e19221119d00da5bec8f19e30500651f6b5a30ae416)). Perhaps I should have used different wallets that are not affected by the bug to keep things “clean”, but this wallet is the easiest to operate, trace, and account for. Adding the 1.8M I sold on Dec 7, I sold a total of 16.4M during all these times. 4. I put everything else aside, started investigating the code and asked around the team for help. See below examples across Dec 10 and 11. Note that it is not my responsibility to do this. My involvement with Harmony has nothing to do with fixing its protocol code. Screenshots available upon request, ([1](https://drive.google.com/file/d/1Lt67rXFJfpcfxe81twBtF5r53R1yG4Ix/view?usp=drive_link), [2](https://drive.google.com/file/d/1U9jGtCkPlJm39irRMJu3kQTmRBtLBvWJ/view?usp=drive_link)) > [Private chat: Aaron Li, Sun Hyuk An] > > (December 10, 6:57pm) > > **Aaron**: are you still syncing with protocol development and recent changes? > > (December 10, 10:51pm) > > **Sun**: I am not... Stephen has told me to only focus on the voice ai. > [Private chat: Aaron Li, Diego Nava] > > (December 11, 5:07am) > > **Aaron**: Diego - is there a place showing recent changes (with dates) on consensus and staking code? Are validator nodes self-updating and automatically incorporating these changes? I am checking some security issues > > (5:41am) **Diego**: hey! mmm about dates the only think that comes to my mind is the GH relases page, but we don't really have a formal tracking of that unfortunately. if its a consesus change or staking will most likely come with a hard-fork and they will be therefore obliged to incroparte those changes. After Hip30 the latest update and hardfork all the validators are running the same and latest release. They don'y self update they have to run the update manually > > (5:42am) do you need help with debugging something, whate security issue you think we might have issues with? > > (5:44am) we can figure out date using the discord notification channel, if it was a hardfork we can find the relase there and look at the changes. > > (5:54am) **Aaron**: thanks, let me check and confirm first. Do we have a way to apply emergency patch? We might also need to build some tools to evaluate the impact > > (6:07am) **Diego**: Not really! the last time we debugged a security issue we pushed a fix disguised as another fix so users won't exploited it until all validators had upgraded it. We still have 49% voting power in our side though so we can push any fix asap and then wait for the rest of the validators to apply it on their side 5. On Dec 12, I finished examining most of undelegation payout code and found a temporary solution. After I confirmed undelegation payouts are continuing for every epoch, I alerted Casey and asked him to do an emergency patch. This was the third time this issue was brought up. [Screenshot available upon request](https://drive.google.com/file/d/1HbFjpMTNszw6OQTiVYureXa5jNnRIG0a/view?usp=sharing) > [Group chat: Aaron Li, Li Jiang, Casey Gardiner, Stephen Tse] > > (December 12, 2:13pm) > > **Aaron**: *@caseyga* we need to do an emergency patch > > (2:13pm) **Casey**: Where? > > (2:14pm) **Aaron**: > > ![2023-12-12-emergency-attached-1](https://storage.googleapis.com/harmony-infinite-minting-files/2023-12-12-emergency-attached-1.jpg) > > change to == > > i still haven't found the root cause but that will stop things getting worse for now > > ![2023-12-12-emergency-attached-2](https://storage.googleapis.com/harmony-infinite-minting-files/2023-12-12-emergency-attached-2.jpg) > > > (2:15pm) **Casey**: ~~is that a smart contract?~~ (Note: quickly deleted) > > (2:15pm) **Casey**: ah 6. On the same day, I wrote [some scripts](https://github.com/polymorpher/harmony-mint-bug-scan) and generated a report of the list of affected accounts, validators, the exact amount being repeatedly paid out, and since which epoch the issue started occurring. The damage is considerable, but “small” from the perspective of the whole tokenomics. [Screenshot available upon request](https://drive.google.com/file/d/1dWituolfMMjZ8ewrTVgruez9xOTPC3z5/view?usp=drive_link) > [Group chat: Aaron Li, Li Jiang, Casey Gardiner, Stephen Tse] > > (December 12, 6:15pm) > > **Aaron**: good news is only 3 validators (out of all registered validators) are affected, the issue only started to happen recently, and the impacted amount are sufficiently "small" to not cause significant damage > > (Attached file: [validator-scan-result.txt](https://storage.googleapis.com/harmony-infinite-minting-files/2023-12-12-stats-validator-scan-result.txt)) > > ![2023-12-12-stats-attached-1](https://storage.googleapis.com/harmony-infinite-minting-files/2023-12-12-stats-attached-1.jpg) > > ![2023-12-12-stats-attached-2](https://storage.googleapis.com/harmony-infinite-minting-files/2023-12-12-stats-attached-2.jpg) > > ![2023-12-12-stats-attached-3](https://storage.googleapis.com/harmony-infinite-minting-files/2023-12-12-stats-attached-3.jpg) > 7. Casey did not do an emergency patch right away. He chose to analyze the code more thoroughly with other engineers in the Harmony team so the root cause can be found and fixed. It took me quite a while (1+ day) to convince him that this is not a display or reporting issue, that extra ONEs are being repeatedly minted. We went through the code and the problem together, identified multiple issues, and eventually Casey replicated the bug on a local net. We confirmed the root cause and a principled patch at around midnight Dec 14. See attached Telegram chatlog. Casey decided to do the patch inconspicuously and schedule it to be effective after 7-14 days. 8. On Dec 14 at around 1pm, I met Stephen at Harmony office. I was surprised to learn that Stephen is under the impression that the problem is only “a few thousand dollars” being minted, based on the information Casey provided to him immediately before my meeting. I advised Stephen that the actual damage is $50k-$75k per day (not including mine) and more than $1M worth of ONEs were already minted. Stephen freaked out, decided to cancel all plans for the day, so Casey can join the meeting and discuss immediate actions we can take. Stephen asked me how I found out about the bug. I told him that I was helping out a validator (H1 Validator) to get elected (which was disclosed to him a few weeks ago with details), and I eventually undelegated my stake when H1 Validator decided to give up. This somehow led to my account getting paid with undelegated amount repeatedly. 9. A long, emergency meeting was held between Casey, Stephen and me. We came up with a better estimation of the damage and its upper limits. It was agreed that an emergency patch is to be scheduled within 3 days. We opined on what to do with the funds but did not reach any decision. Casey and Stephen were both nice enough to say that I should be able to decide what I want to do with the funds under my account, but we agreed we want to do the right thing. A discord announcement was made later that day. 10. During the day, Casey asked whether [0x18a5af69…180f34b75d7a](https://explorer.harmony.one/address/0x18a5af69dfc02dc950f7534ae97a180f34b75d7a?activeTab=0) is an account I control. I answered affirmatively. He seemed surprised by the amounts. He asked me where I am transferring the funds to. I explained that I am spreading them into multiple wallets so the balance wouldn’t appear too big and cause unnecessary panics. When Stephen came in, I explained to him that my wallet may have caught about half of the funds - in case he forgot my delegated numbers - so the problem might be much smaller than he thought. 11. Later at night on Dec 14, Casey asked me whether I sold some. I answered affirmatively and provided him with the exact amounts. He became upset and started calling it “malicious”. 12. On Dec 15, Stephen, Casey, Adam (Casey’s friend), had lunch together. After lunch, I met Adam and Casey without Stephen. Surprisingly, Casey began a train of accusations of theft and insults. I was questioned about the sale and was called “unethical”, a “thief” for selling hacked / exploited ONEs, despite the fact that I legitimately owned more than what I sold. I was told all my accounts were “blacklisted”, even including exchange accounts, and the only thing I could do was to burn them by sending them to the 0x0 address. This is very different to what we discussed a day before, and I do not fully agree burning is the only and right approach. However, when I said we should ask the validator and the community first, and discouraged him from using blacklist as it is a form of censorship, Casey accused me of stalling, being unethical, and doing exactly what criminals do. I asked him to calm down as I saw the erratic behavior could be a result of extreme stress and anxiety. Nonetheless, the meeting did not go well. 13. Later that day, I checked my various wallets and moved the majority of the funds to a custom designed smart contract wallet, which is censorship resistant - i.e. unblockable. I disagree that rash decisions should be made on this, or that he alone should make unilateral decisions without input from others and thorough reasoning. I also sent him a long message explaining I am open to burning all the funds, if he has to insist, as I believe we all want the project to become better, but we may differ in our approaches, methods, and principles. 14. Unfortunately this measure led to more misunderstanding. Casey believed I sold 600000 ONE more after our meeting and went ballistic on my honesty and ethic for selling “stolen” funds. I explained how an exchange wallet works, that the purpose was for testing blacklist, and where the funds came from. Casey realized his mistake and apologized. But soon things escalated to the level that he started filing reports at the FBI and IRS via his other organizational affiliation channels. He claimed that I have no integrity, no one trusts me (15+ validators, people in the office, and remote engineers), and all I was trying to do is to hide, steal, and take money, and he will do everything in his power to burn me, and he will do everything to run me away from Harmony. And he will ban me from any Ethereum Foundation and ETHDenver activities. He claimed I was not forthcoming with him and was hiding information from him, despite I told him numerous times that I didn’t know him well enough to disclose information that is too personal, or to simply put it - none of his business to know. I also raised concern that his initial behaviors made me really doubt his intentions and whether I could trust him - e.g. not responding when the issue was first brought up on Dec 7, or trying to give Stephen incorrect information to minimize problems on his side, among other things. 15. Neither of us had any sleep last night or today. At this time, I am avoiding further escalation of unnecessary conflict and ceased communication. I believe the project is at a stage that needs unification of efforts more than ever. The last thing people want to see are infighting, theatrics, or politics. In my view, integrity and trustworthiness are shown by actions, thoughtfulness, and results, tested by temptations over and over, not by blindly obliging some doctrine. The people will be the best judge for that. Despite everything Casey has said, I still believe he is a capable engineer and Harmony could benefit from some of his technical participation. I hope that he will become an inspiring leader one day. 16. I think it is best to keep [private messages](https://drive.google.com/file/d/1yZ7FdVAvfNvyfiGCF30dbDXqERpy7kdg/view?usp=sharing) private. I am leaving the restricted link here as a reminder that we all say stupid things from time to time. Best to move on and stay positive. 17. Despite Casey’s use of blacklist censorship, I transferred out 26.4M ONE from the six unblockable wallets and started the burning process after consensus was reached among recovery custodian validators, partners, and selected independent validators on what to do with the fund. [https://explorer.harmony.one/tx/0xd831fab50ef5835737284e1d3dd00e5ff51c4a255b45718656a12920ee93471a](https://explorer.harmony.one/tx/0xd831fab50ef5835737284e1d3dd00e5ff51c4a255b45718656a12920ee93471a) [https://explorer.harmony.one/tx/0x65782dcbf1c244a31ded3ecf127fdaa482fa6c3d47e8eb00435263cc2f5569f1](https://explorer.harmony.one/tx/0x65782dcbf1c244a31ded3ecf127fdaa482fa6c3d47e8eb00435263cc2f5569f1) [https://explorer.harmony.one/tx/0xb15c67c4c1792577472b09620afdf1eeef32ed40880eed69b249b332e6e0a2ce](https://explorer.harmony.one/tx/0xb15c67c4c1792577472b09620afdf1eeef32ed40880eed69b249b332e6e0a2ce) [https://explorer.harmony.one/tx/0x15116818e367112158c34a153267f886789a002c1076f2481750281fdd6f1e37](https://explorer.harmony.one/tx/0x15116818e367112158c34a153267f886789a002c1076f2481750281fdd6f1e37) [https://explorer.harmony.one/tx/0x0f6bd776a52d2a29ff656a89ab9b864ce7b89a1fc4669ea347f9b9b9f8b422f9](https://explorer.harmony.one/tx/0x0f6bd776a52d2a29ff656a89ab9b864ce7b89a1fc4669ea347f9b9b9f8b422f9) [https://explorer.harmony.one/tx/0x7b8db7ed41f76225a73fa5d5cef6c111d6cd6186fd4e7ec7e130490bcf0cae4d](https://explorer.harmony.one/tx/0x7b8db7ed41f76225a73fa5d5cef6c111d6cd6186fd4e7ec7e130490bcf0cae4d) 18. A total of 12.8M ONE is stuck in my wallet [0x18A5…5d7a](https://explorer.harmony.one/address/0x18A5Af69DfC02dc950f7534Ae97A180F34B75d7a) that had been erroneously receiving minted ONEs. The current implementations of whitelist and blacklist fails to allow the wallet from burning the ONEs. As soon as the issue is addressed by the team, the burning of this 12.8M may proceed. Update: the issue was fixed on Dec 17 at 1:30pm. The 12.8M is now burned: https://explorer.harmony.one/tx/0x6b21695d0a9f71a0d4877a18aa94c9add5fe99f2464d70232d9f4bc27e8e322b 19. Another 5.6M ONE is planned to be provided from [0x14b6e..ab1d61](https://explorer.harmony.one/address/0x14b6e8df446e81c7f2c69ba1573197ccebab1d61) after the [undelegation](https://explorer.harmony.one/tx/0xbb734439227867e53dd41e19221119d00da5bec8f19e30500651f6b5a30ae416) from modulo.so validator is complete. **Update Jan 18, 2024**: the 5.6M ONE is now burned: [https://explorer.harmony.one/tx/0x450a7292c23ffe2ce9c303476f68d344c4298e8d7561fe51dab330b637ff4a2d](https://explorer.harmony.one/tx/0x450a7292c23ffe2ce9c303476f68d344c4298e8d7561fe51dab330b637ff4a2d) 20. The remaining 6.4M ONE will be burned by two wallets: - 3.4M is [already burned](https://explorer.harmony.one/tx/0xffa078bbcf0f9b81a6a966bf74f41acac42320e306005a9af2257f099de34afb) by [0xF366b6E4bb3b5F9c8219e9281dda1f849eFdD35C](https://explorer.harmony.one/address/0xf366b6e4bb3b5f9c8219e9281dda1f849efdd35c) - 3M is to be undelegated from [0x18A5…5d7a](https://explorer.harmony.one/address/0x18A5Af69DfC02dc950f7534Ae97A180F34B75d7a) when the blacklist on the wallet is removed, then the amount will be burned ~5 days later when undelegation completes. ( Update: the undelegation is already [initiated](https://explorer.harmony.one/tx/0x775f70f9dc22c567e3fa50f8add6173a2a5ac7103bd6d69ecf97e071d3452e1b) on Dec 19 at 9:20am) **Update Jan 18, 2024**: the 3M ONE is now burned [https://explorer.harmony.one/tx/0xd7aebb2cd146beba76feced64ffe87888ec5885b15144c04a5e5185925dd3cf8](https://explorer.harmony.one/tx/0xd7aebb2cd146beba76feced64ffe87888ec5885b15144c04a5e5185925dd3cf8) 21. As of Jan 18, 2024, all 51.2M ONE under my control are burned. ## Appendix: Verifying extra ONE minted to `0x18A5…5d7a` ### Undelegation Analysis My wallet [0x18A5…5d7a](https://explorer.harmony.one/address/0x18A5Af69DfC02dc950f7534Ae97A180F34B75d7a) had two undelegations at different epoch times, so the sum needs to be computed separately for these two. First, using my scan tool at the block corresponding to epoch 1732 (when bad delegation data still existed on blockchain state), we can find the line corresponding to this wallet. A precompiled output is already provided in [`don.txt`](https://github.com/polymorpher/harmony-mint-bug-scan/blob/b6a211aca65d462723e67aae71a47d0d38a42230/don.txt#L4) ``` Bad delegations from validator one1qv82h8dzjmm09tqwdwe3evjgmzz84fqmqskkzk amount per epoch 4399999.999999991 minted sum 51199999.999999896 since 1713 bad delegations { "amount":0, "delegator-address":"one1rzj676wlcqkuj58h2d9wj7scpu6twht6qqvw54", "reward":1.95620373278941e+22, "undelegations":[{"amount":1.4e+24,"epoch":1711},{"amount":3e+24,"epoch":1713}] } ``` Where `one1rzj676wlcqkuj58h2d9wj7scpu6twht6qqvw54` is the equivalent address in bech32 for 0x18A5Af69DfC02dc950f7534Ae97A180F34B75d7a. Note that epoch 1732 does not create extra payout. This is because the payout was to be made at epoch 1733, at which time the emergency patch became effective and the bug became fixed. Undelegation 1.4e+24 starts to accumulate ONEs from epoch 1718 (=1711+7), until 1731 (inclusive). The sum is (1731 - 1718 + 1) * 1.4e+24 = 19.6e+24 Undelegation 3e+24 starts to accumulate ONEs from epoch 1720 (=1713+7), until 1731 (inclusive). The sum is (1731 - 1720 + 1) * 3e+24 = 36e+24 This gives a total amount of 55.6e+24. Among that, the incorrect amount minted is 55.6e+24 - 1.4e+24 - 3e+24 = 51.2e+24, the subtraction is needed because the first minting for both is for the legitimate undelegation. Therefore, the total amount incorrectly minted is 51.2M (= 51.2e+24 / 1e+18) ### Account Cashflow Analysis Another way to compute is to get the wallet balance at block 50465369, when one undelegation just started two blocks ago, and another undelegation just begun. The balance is 0x3866a48c2c66f40fb, i.e. 65.02586617113236. This roughly equals to the present balance of the account (or say, block 51221232). The historical balance can be queried via an RPC call to an archival node, for example: ``` curl --location 'https://a.api.s0.t.hmny.io/' \ --header 'Content-Type: application/json' \ --data '{ "jsonrpc": "2.0", "method": "eth_getBalance", "params": [ "0x18A5Af69DfC02dc950f7534Ae97A180F34B75d7a", "0x3020a59" ], "id": 1 }' ``` We then can look at all transactions performed by this account from the block 50465369 until now (or block 51221232). Since there has been no smart contract transactions, we can simply sum up the value of each transaction going out from the wallet. This can be easily inspected using the explorer. The sum is 4400000 * 9 + 12800000 + 1400000 + 400000 + 1000000 + 400000 = 55.6M. Therefore, the extra minted amount can be computed by subtracting the legitimate undelegation payouts: 55.6M - 3M - 1.4M = 51.2M ### Epoch Boundary Balance Difference Analysis We can compute the difference of account balance at each epoch boundary block. In such block, the undelegation payout is made. Unless there is any transaction affecting this account's balance that exactly occurred in such block, the sum of the differences would give us the total amount of extra minted ONE, plus the payout for legitimately undelegated ONE. It is easy to verify the wallet does not have any such transaction at epoch boundary block - one can simply iterate through all transactions in such blocks and check their side effects. I created a tool to automate the process of computing epoch boundary block number for given epochs, the differences in balance. The tool can be found in [balance-check.ts](https://github.com/polymorpher/harmony-mint-bug-scan/blob/main/balance-check.ts), and the result is in [aaron-extra-mint.txt](https://github.com/polymorpher/harmony-mint-bug-scan/blob/main/aaron-extra-mint.txt): ``` Epoch 1713 block 50495487 balance-diff 0 Epoch 1714 block 50528255 balance-diff 0 Epoch 1715 block 50561023 balance-diff 0 Epoch 1716 block 50593791 balance-diff 0 Epoch 1717 block 50626559 balance-diff 0 Epoch 1718 block 50659327 balance-diff 1400000000000000000000000 Epoch 1719 block 50692095 balance-diff 1400000000000000000000000 Epoch 1720 block 50724863 balance-diff 4400000000000000000000000 Epoch 1721 block 50757631 balance-diff 4400000000000000000000000 Epoch 1722 block 50790399 balance-diff 4400000000000000000000000 Epoch 1723 block 50823167 balance-diff 4400000000000000000000000 Epoch 1724 block 50855935 balance-diff 4400000000000000000000000 Epoch 1725 block 50888703 balance-diff 4400000000000000000000000 Epoch 1726 block 50921471 balance-diff 4400000000000000000000000 Epoch 1727 block 50954239 balance-diff 4400000000000000000000000 Epoch 1728 block 50987007 balance-diff 4400000000000000000000000 Epoch 1729 block 51019775 balance-diff 4400000000000000000000000 Epoch 1730 block 51052543 balance-diff 4400000000000000000000000 Epoch 1731 block 51085311 balance-diff 4400000000000000000000000 Epoch 1732 block 51118079 balance-diff 0 Epoch 1733 block 51150847 balance-diff 0 Epoch 1734 block 51183615 balance-diff 0 Total sum (including legitimate undelegation) 55600000 ``` Again, subtracting legitimate payouts (4.4M) for undelegation from the sum (55.6M), we arrive at the same number, 51.2M. ## Change Notes - Jan 18, 2024: updated the links to burning all remaining amount - Dec 22, 2023: added the one missing validator to the list of impacted validators, and 33 delegators with the sum of 6.8M ONE - Dec 20, 2023: initial publication