# Mira --- ### Main types: * Task * Graph * Runner * Store --- ### Task ```py class Foo(Task): @staticmethod def work(config, store): .... ``` --- ### Task (ctd...) ```py class Bar(Task): def outputs(self): return ['bar'] @staticmethod def work(config, store): data = store['foo'] # no deserialize ... store['bar'] = pd.DataFrame(...) # no serialize ``` --- ### Task (ctd...) ```py class Scatter(ParallelTask): def outputs(self): return ["sim"] def chunks(self): return itertools.chunked(range(10), 3) @staticmethod def work(chunk, config, store): ... store[f"sim/{Scatter.key(chunk)}"] = ... ``` --- ### Graph ```py # kf_pipelines/aera/benqi_autogov/trigger_pipeline.py def make_graph(): g = Graph() loanbook = g.add(LoanBook) prices = g.add(Prices) market_context = g.add(MarketContext) sim = g.add(ParallelSim).after(loanbook, prices, market_context) g.add(UploadResults).after(sim) return g ``` --- ### Runner * LocalRunner * RemoteRunner ```py RemoteRunner(config, "my-experiment").run(g) ``` ```py LocalRunner(config, "my-experiment").run(g) ``` --- ### Store * LocalStore * RemoteStore * MemoryStore (for testing) ```py >>> from backend.mira import RemoteStore >>> store = RemoteStore('demo') >>> store['metadata'] ... ``` --- ### Store (ctd...) ```py >>> from backend.mira import RemoteStore >>> store = RemoteStore('multi-gbm', user='nick') >>> store['prices'] ``` --- ### cli ```sh % mira --help Usage: mira [OPTIONS] COMMAND [ARGS]... Options: --help Show this message and exit. Commands: delete invalidate list metadata read ``` --- ### cli (ctd...) ```sh % mira list test-basic better-graph aera_benqi demo playground % mira list -u victorxu aera_benqi ``` --- ### metadata ```sh % mira metadata demo name demo link https://kf-dev.endpoints.gauntlet-sim.cloud.goog/pipeline/#/runs/details/808c9e47-bbd7-4ff3-8549-6b3dd03227f2 branch mira-key-hash commit 7e23329863aacd23d9279abd91c7c52fbb666f83 dirty True lastran 2022-08-13 21:34:24.136715 runner sandeepgupta ``` --- # END ---
{"metaMigratedAt":"2023-06-17T06:52:34.558Z","metaMigratedFrom":"YAML","title":"Mira","breaks":true,"contributors":"[{\"id\":\"4f552fb1-9945-4de4-97bf-04dd414ff08b\",\"add\":2582,\"del\":295}]"}
    141 views