# OPP ## Local env setup 1. Clone the repo 2. initialize the dependency submodules as ``` git submodule update --init <sub_module> ``` 3. build new version of the database image, required for the opp-user-management-svc application `docker build -f mock-dependencies/Dockerfile.opp-grant-service-db mock-dependencies/ -t opp-grant-service-db:v0.0.1-1-c0f0f1a` 2. build new version of the opp-user-management-svc image `docker build -f opp-grant-service-core/Dockerfile -t opp-grant-service-core:v0.0.1-c0f0f1a opp-grant-service-core/` ## IMP links ### Git Repo https://git.rakuten-it.com/projects/ONEPOINT/repos/opp-grant-service/browse ### Jira board https://jira.rakuten-it.com/jira/secure/RapidBoard.jspa?rapidView=22538 ### Team Mowgli Weekly https://confluence.rakuten-it.com/confluence/display/ONEPOINT/Weekly+2020.01+Team+Mowgli ### JPP Bridge https://confluence.rakuten-it.com/confluence/pages/viewpage.action?pageId=2273018789 ### OPP Architecture https://confluence.rakuten-it.com/confluence/display/ONEPOINT/One+Point+System+Overview ### OPP APIs https://confluence.rakuten-it.com/confluence/display/ONEPOINT/APIs https://confluence.rakuten-it.com/confluence/display/ONEPOINT/Grant+APIs ### OPP V4 Compatibility https://confluence.rakuten-it.com/confluence/display/ONEPOINT/JPP+V4+Grant+%3C-%3E+OPP+Grant+Compatibility ### OPP-JPP endpoints compatibility https://confluence.rakuten-it.com/confluence/display/ONEPOINT/Compatibility+-+JPP+to+OPP+API+Transition ### backward compatibility with Legacy https://confluence.rakuten-it.com/confluence/display/ONEPOINT/OPP+Grant+-+Backward+Compatibility+with+Legacy ### GPP Architecture https://confluence.rakuten-it.com/confluence/display/GPPGB/GPoint+System+Architecture https://confluence.rakuten-it.com/confluence/display/GPPGB/Aldebaran+PMS+API+-+Auto+Recovery https://confluence.rakuten-it.com/confluence/display/GPPGB/Manual+Publish+of+Recovery+Info+to+RabbitMQ The list of properties for each applications are available in the confluence “main” page of the corresponding application: https://confluence.rakuten-it.com/confluence/display/ONEPOINT/Authorization : Token Service https://confluence.rakuten-it.com/confluence/pages/viewpage.action?pageId=2229035651 : Point Type and Currency Service https://confluence.rakuten-it.com/confluence/display/ONEPOINT/Client+Management+APIs : Client Service https://confluence.rakuten-it.com/confluence/display/ONEPOINT/User+Management+APIs : User Service #### API Design: GPP-JPP API Mapping: https://confluence.rakuten-it.com/confluence/display/GPPGB/GPP-JPP+API+Mapping GPP-JPP API Param Mapping: https://confluence.rakuten-it.com/confluence/display/GPPGB/GPP-JPP+API+Param+Mapping JPP Invest API: https://confluence.rakuten-it.com/confluence/display/point/invest+-+PointAPI JPP V4 Grant <-> OPP Grant Compatibility: https://confluence.rakuten-it.com/confluence/display/ONEPOINT/JPP+V4+Grant+%3C-%3E+OPP+Grant+Compatibility same invest API in JPP can be used to grant - api call with new ref/sub key update grant - api call with existing ref/sub key and points > 0 cancel grant - api call with existing ref/sub key and points = 0 ``` <data> <easy_id>31601590</easy_id> <group_id>47</group_id> <reason_id>91</reason_id> <tx_key>testRcashMultival</tx_key> <tx_sub_key/> <exec_seq>1</exec_seq> <res_time>2020-05-13 15:03:17</res_time> <result_code>D30</result_code> </data> <confirmation_key>a76278224957eeedf21911edd3819669</confirmation_key> </Root> ``` ### Meeting notes with Rohit 1. add client mapping params doc link 2. add note for sub key ### In the eventMetadata .. we have a eventPublishedTime attribute… which I presume is the time when OPP grant sends the event to kafka… in the payload, we have a grantTime attribute which I presume is the time the grant was done in OPP. I need clarification on the following points.. when the eventType is GRANT or GRANT_UPDATE, we need to pass the grantTime received in payload as tx_time for the legacy invest api..? when the eventType is GRANT_CANCEL, we need to pass the eventPublishedTime as tx_time for the legacy invest api.? The reason I am asking this is because the grantTime is not mandatory for cancel event, but eventPublishedTime is mandatory for all the event types. ## JPP API call ### install dependencies ``` git submodule update --init --recursive ``` ### Docker image build ``` docker build -f mock-dependencies/Dockerfile.bridge-db -t <<give same name as docker-compse>> . ``` ### Steps to get started 1. git clone <opp-jpp-grant-bridge-service-repo> 2. Import as gradle project in Intellij 3. Run the cmd under root of this repo git submodule update --init --recursive 4. Run docker-compose up Check if 3 processes are running with docker ps with 3 different containers for Kafka, Zookeeper, Bridge_DB 5. Run the application with gradle task Run ### Steps to run 1. Run docker-compose up in /opp-grant-bridge-service dir. This will create containers for Kafka, Zookeeper, bridge_db 2. Do docker exec -it <Kafka-container-id> bash to login 3. Run inside kafka container: ```kafka-console-producer.sh --broker-list localhost:9092 --topic opp-jpp-bridge-grant-tx``` This will start Kafka producer. 4. Publish event for ex for STD grant : ```{"transactionId":"fea9a3b2-e985-4c63-87a9-c396b1205e11","oppUserId":"272ea115-4f58-49da-9d66-fe92e876aae9","legacyTarget":"JPP","legacyUserId":{"easyId":2010},"eventType":"GRANT","sequence":0,"traceId":"624f4a3c-e7f9-4967-b83b-1d639a878834","eventPublishedTime":"2020-06-01T05:38:15.538+0000","payload":{"clientId":"client2","subId":"sub2","pointType":"STDSS","points":1000,"shopId":123,"shopName":"Central Universal Department Store","shopUrl":"https://www.tsum.ru/","itemName":"Pack of 4 shot glasses for vodka \"Wild boar\"","itemUrl":"https://www.tsum.ru/product/00034188-nabor-iz-4-kh-ryumok-dlya-vodki-dikii-kaban-tsar-prozrachnyi/","itemPrice":69850.0,"grantTime":"2020-06-01T05:40:15.538+0000"}}``` in the same producer This event should be consumed in bridge ### Cassandra config to docker file ``` cassandra: image: jpp-bridge-db hostname: cassandra ports: - 9042:9042 ``` ### Insert Data to Client Table ``` INSERT INTO opp_bridge.client_sub_id_lookup(client_id,sub_id,group_id,reason_id,created_at,updated_at) VALUES ('client1', 'sub1', 47, 263, '1970-06-01T05:40:15.538+0000' , '2020-06-01T05:40:15.538+0000') ; INSERT INTO opp_bridge.client_sub_id_lookup(client_id,sub_id,group_id,reason_id,created_at,updated_at) VALUES ('client2', 'sub2', 47, 91, '1970-06-01T05:40:15.538+0000' , '2020-06-01T05:40:15.538+0000') ; ``` ### Create Index ```CREATE INDEX event_status_index ON "opp_ems"."event_destination_status" (status); ### Update Status UPDATE "opp_ems"."event_destination_status" SET status = 'SUCCESS' WHERE event_id = 6c9b13e2-b4f3-11ea-b3de-0242ac130016 AND currency = 'JPY' AND service = 'GRANT' AND created_at = '2020-07-03 00:00:00+0000'; ### Select with status ```SELECT * FROM "opp_ems"."event_destination_status" WHERE currency = 'JPY' AND service = 'GRANT' AND status = 'NEED_RECOVERY';``` currency, service, event_id, destination, status SELECT currency, service, event_id, destination, status FROM "opp_ems"."event_destination_status" WHERE currency = 'JPY' AND service = 'GRANT' AND status = 'NEED_RECOVERY'; ### Start Kafka Producer ``` kafka-console-producer.sh --broker-list localhost:9092 --topic opp-jpp-bridge-grant-tx ``` ### Start Kafka Consumer ``` kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic opp-grant-bridge-ack --from-beginning ``` 1. ### Standard Grant ``` {"eventId":"5c9b13e2-b4f3-11ea-b3de-0242ac130004","transactionId":"aea9a3b2-e785-4c63-87a9-c396b1205e11","oppUserId":"272ea115-4f58-49da-9d66-fe92e876aae9","legacyTarget":"JPP","legacyUserId":{"easyId":2010},"eventType":"GRANT","sequence":0,"traceId":"624f4a3c-e7f9-4967-b83b-1d639a878834","eventPublishedTime":"2020-06-01T05:38:15.538+0000","payload":{"clientId":"client2","subId":"sub2","pointType":"STD","points":1000,"shopId":123,"shopName":"Central Universal Department Store","shopUrl":"https://www.tsum.ru/","itemName":"Pack of 4 shot glasses for vodka \"Wild boar\"","itemUrl":"https://www.tsum.ru/product/00034188-nabor-iz-4-kh-ryumok-dlya-vodki-dikii-kaban-tsar-prozrachnyi/","itemPrice":69850.0,"grantTime":"2020-06-01T05:40:15.538+0000"}} ``` ### Update Event ``` {"eventId":"5c9b13e2-b4f3-11ea-b3de-0242ac130004","transactionId":"fea9a3b2-e985-4c63-87a9-c396b1205e11","oppUserId":"272ea115-4f58-49da-9d66-fe92e876aae9","legacyTarget":"JPP","legacyUserId":{"easyId":2007},"eventType":"GRANT_UPDATE","sequence":0,"traceId":"624f4a3c-e7f9-4967-b83b-1d639a878834","eventPublishedTime":"2020-06-01T05:38:15.538+0000","payload":{"clientId":"123","subId":"123-sub","pointType":"STD","points":1000,"shopId":123,"shopName":"Central Universal Department Store","shopUrl":"https://www.tsum.ru/","itemName":"Pack of 4 shot glasses for vodka \"Wild boar\"","itemUrl":"https://www.tsum.ru/product/00034188-nabor-iz-4-kh-ryumok-dlya-vodki-dikii-kaban-tsar-prozrachnyi/","itemPrice":69850.0,"grantTime":"2020-06-01T05:40:15.538+0000"}} ``` ### 2.Term Grant #### old ``` {"transactionId":"fea9a3b2-e985-4c63-87a9-c396b1205e02","oppUserId":"272ea115-4f58-49da-9d66-fe92e876aae9","legacyTarget": "JPP", "legacyUserId":{ "easyId":2324},"eventType":"GRANT","sequence":0,"traceId":"624f4a3c-e7f9-4967-b83b-1d639a878809","eventPublishedTime":"2020-05-24T05:38:15.538+0000","payload":{"clientId":"123","subId":"123-sub","pointType":"TERM","points":698,"expiryTime":"2020-06-02T15:22:02.000+0000","shopId":123,"shopName":"Central Universal Department Store","shopUrl":"https://www.tsum.ru/","itemName":"Pack of 4 shot glasses for vodka \"Wild boar\"","itemUrl":"https://www.tsum.ru/product/00034188-nabor-iz-4-kh-ryumok-dlya-vodki-dikii-kaban-tsar-prozrachnyi/","itemPrice":69850.0,"grantTime":"2020-05-24T05:38:15.538+0000"}} ``` #### new event **Worked for Demo of Acknowledgement Event:** ```{"eventId":"5c9b13e2-b4f3-11ea-b3de-0242ac130004","transactionId":"aea9a3b2-e785-4c63-87a9-c396b1205e11","oppUserId":"272ea115-4f58-49da-9d66-fe92e876aae9","legacyTarget":"JPP","legacyUserId":{"easyId":2010},"eventType":"GRANT","sequence":0,"traceId":"624f4a3c-e7f9-4967-b83b-1d639a878834","eventPublishedTime":"2020-06-01T05:38:15.538+0000","payload":{"clientId":"client2","requestKey": "requestKey133232","subId":"sub2","pointType":"TERM","points":1000,"expiryTime":"2020-06-02T15:22:02.000+0000","shopId":123,"shopName":"Central Universal Department Store","shopUrl":"https://www.tsum.ru/","itemName":"Pack of 4 shot glasses for vodka \"Wild boar\"","itemUrl":"https://www.tsum.ru/product/00034188-nabor-iz-4-kh-ryumok-dlya-vodki-dikii-kaban-tsar-prozrachnyi/","itemPrice":69850.0,"grantTime":"2020-06-01T05:40:15.538+0000"}}``` ```{ "eventId": "5c9b13e2-b4f3-11ea-b3de-0242ac130004", "transactionId": "aea9a3b2-e785-4c63-87a9-c396b1205e11", "oppUserId": "272ea115-4f58-49da-9d66-fe92e876aae9", "legacyTarget": "JPP", "legacyUserId": { "easyId": 2010 }, "eventType": "GRANT", "sequence": 0, "traceId": "624f4a3c-e7f9-4967-b83b-1d639a878834", "eventPublishedTime": "2020-06-01T05:38:15.538+0000", "payload": { "clientId": "client2", "subId": "sub2", "pointType": "TERM", "points": 1000, "expiryTime": "2020-06-02T15:22:02.000+0000", "shopId": 123, "shopName": "Central Universal Department Store", "shopUrl": "https://www.tsum.ru/", "itemName": "Pack of 4 shot glasses for vodka \"Wild boar\"", "itemUrl": "https://www.tsum.ru/product/00034188-nabor-iz-4-kh-ryumok-dlya-vodki-dikii-kaban-tsar-prozrachnyi/", "itemPrice": 69850.0, "grantTime": "2020-06-01T05:40:15.538+0000" } }``` ##### After Schema change ```{"eventId":"bd126f4b-250e-4acf-9fd5-76bff7cc09c8","eventTime":"2020-06-01T16:22:02.000+0000","service":"GRANT","currency": "JPY","eventType":"GRANT","traceId":"7773527b-49cb-4eef-ba89-1d61eca2e30f","eventPublishedTime":"2020-06-02T15:22:02.000+0000","payload":{"txId":"3d30fd6f-60f0-47be-8931-a0c466e7503c","oppUserId":"f088abb0-0814-43ae-95ae-5d3bb8097dfc","easyId":1234,"sequence":0,"requestKey":"requestKey133232","clientId":"client2","subId":"sub2","pointType":"STD","points":100,"shopId":"example-shop","descriptionText":"Central Universal Department Store","descriptionUrl":"https://www.tsum.ru/","itemName":"Pack of 4 shot glasses for vodka Wild boar","itemUrl":"https://www.tsum.ru/product/00034188-nabor-iz-4-kh-ryumok-dlya-vodki-dikii-kaban-tsar-prozrachnyi/","itemPrice":69850.0,"orderTime":"2020-06-01T05:40:15.538+0000","grantTime":"2020-06-01T05:40:15.538+0000","fixTime":"2020-06-01T05:40:15.538+0000"}} ``` ##### Wrong event input to test traceId bug ```{"eventId":"bd126f4b-250e-4acf-9fd5-76bff7cc09c8","eventTime":"2020-06-01T16:22:02.000+0000","service":"GRANT","currency": "JPY","eventType":"GRANT","traceId":"7773527b-49cb-4eef-ba89-1d61eca2e30f","eventPublishedTime":"2020-06-02T15:22:02.000+0000","payload":{"txId":"3d30fd6f-60f0-47be-8931-a0c466e7503c","sequence":0,"requestKey":"requestKey133232","clientId":"client2","subId":"sub2","pointType":"STD","points":100,"shopId":"example-shop","descriptionText":"Central Universal Department Store","descriptionUrl":"https://www.tsum.ru/","itemName":"Pack of 4 shot glasses for vodka Wild boar","itemUrl":"https://www.tsum.ru/product/00034188-nabor-iz-4-kh-ryumok-dlya-vodki-dikii-kaban-tsar-prozrachnyi/","itemPrice":69850.0,"orderTime":"2020-06-01T05:40:15.538+0000","grantTime":"2020-06-01T05:40:15.538+0000","fixTime":"2020-06-01T05:40:15.538+0000"}} ``` ### Formatting Comment For quick formatting you can either install ktlint clu or use gradle task that we have. ktlintCheck for the check and ktlintFormat for autofixes. To configure IDEA you require ktlint itself. You need to run ktlint applyToIDEA to apply to the IDE itself or ktlint applyToIDEAProject for a specific problems. Refer to official docs for that Beware that you might encounter issues related to IDE caching its configurations. Perhaps better to apply the changes with the IDE turned off, but I am not sure ### Steps for client creation and API registrations 1. msg SRE SAM to give u permissions to Jenkins https://accounts-onecloud.rakuten-it.com/auth/realms/roc/login-actions/authenticate?execution=082bba4d-3ee3-4641-9eaa-3e706a94bc6b&client_id=rns%3Aroc%3Acaas-jenkins&tab_id=EiwdInazzYo 2. Add APIs in https://git.rakuten-it.com/projects/ONEPOINT/repos/opp-client-management-service/browse/db-script/mysql/dml.sql Run the yml file on ur local kubectl apply abc.yml https://jenkins-jpe2.r-local.net/jenkins/job/Tenant/job/esd-point/job/Namespace/ esd-point-opp-env-stg > Apps > multi-stage-build for the build > onepoint-helm-deploy for the deployment > configmap-deploy if you need to refresh the configmap containing the config of the application ### OPP Clients POINT ADMIN - Admin tool <opp-pointadmin> - all currency, point and client management APIs GRANT grant <opp-grant> - add user API fix worker <opp-grant-fixWorker> - fix API expiry worker <opp-grant-expiryWorker> - expire API EMS EMS worker <opp-ems-worker> - add event, update event status API EMS collector batch <opp-ems-collector> - get bucket and update bucket EMS consistency checker <opp-ems-consistencyChecker> - get event by status BRIDGE Bridge <opp-bridge> - get user API