tags: WebHack#43

WebHack #43 Challenges of Global Infrastructure

Talk 1: Rakuten Platform Overview

Challenges:

  • how to keep scaling
  • how can we keep 1 step ahead
  • staying ahead of latest trends in the developer community

Talk 2: "Unclouding" Container Challenges

Challenge#1: Communication Cost
Challenge#2: Day 2 Ops

Talk 3: Kafka & Hadoop in Rakuten

Challenges on Kafka => Solution:

  • Mirroring througput between region or zone - Parallelism
  • Instability or cluster broken => Scale-out than Scale-Up
  • JDK & Cross-Realm Issue => Global KDC & Proper Streaming Solution
  • OOM on Brokers or Zookeeper => Confirm Use-case and Dedicated ZK

Q1: Regarding kafka streaming and spark streaming, what are the major differences? And how should we choose one of them in the actual project?
kafka streaming

  • very real time
  • application
  • pure real time streaming

spark streaming

  • major batch

Q2:For data warehouse, hadoop or hbase, what are the most use cases for these two majoy DB
hadoop

Select a repo