Software, Physics, Data, Mountains

...and other random associations

Projects in GCP Using Central Billing Accounts

Many organizations recognize the benefits of empowering their developers. In a cloud environment, that often means giving developers the ability to create and manage their own infrastructure.

Of course, developers can easily create their own individual or G-Suite GCP accounts. They can take advantage of the free trial that Google Cloud offers. That’s great, and everything’s hunky-dory until the credit runs out. What then?

In this post I describe a really simple way to set up and use centralized billing on GCP… even across external development accounts. Way better than trying to get me to fill out expense reports for infradev!

Develop Spark Apps on Yarn Using Docker

At svds, we’ll often run spark on yarn in production. Add some artful tuning and this works pretty well. However, developers typically build and test spark application in standalone mode… not on yarn.

Rather than get bitten by the ideosyncracies involved in running spark on yarn -vs- standalone when you go to deploy, here’s a way to set up a development environment for spark that more closely mimics how it’s used in the wild.

SSH Tips and Tricks

Notes from a lunch-and-learn talk. It’s a little weak without the narrative, but I’ll post it here for reference anyway.

Agenda:

  • config
  • tunnels
    • forward
    • reverse
  • proxies
  • ssh + tmux = <3
    • ssh, then tmux
    • tmux, then ssh