Gee
1 min readMay 24, 2020

--

Nice comparison! I'm also using both Glue Job and EMR to run Spark applications and just wanted to share some of the downsides I've seen so for with Glue Jobs.

- when using Glue extensions (e.g. DynamicFrame), it's hard to debug Spark application locally. Sure you could spin up a dev endpoint or a notebook but that's still another step.

- when running an application daily with an execution time under 10minutes, we still have to pay for 10 minutes.

--

--

Gee
Gee

Written by Gee

Data/Machine Learning Engineer

No responses yet