PythonWise

If it won't be simple, it simply won't be. [Hire me, source code] by Miki Tebeka, CEO, 353Solutions

Sunday, May 14, 2017

scikit-learn Compatible Pipeline Steps

A client wanted a way to create a pipeline of transformations on DataFrames. Since they already work with scikit-learn, they were familiar with Pipelines. It took very little code to create a base class for a pipeline step that will work with DataFrames.


If you find this interesting, you might want to hire me ;)

Wednesday, March 29, 2017

Color Log Lines

There are several log handler out there the color log lines. However I prefer to leave the log as is and when I want colors I pipe the log via a utility that does that. Coloring log lines by levels can be easily done with a little awk.

Here's an example how it looks

Monday, January 09, 2017

353Solutions - 2016 in Review

Happy new year!

Let's see how 353solutions did in 2016. We'll start with the numbers and then some insights and future goals.

Numbers
  • Total of 193 work days (days where customer were billed)
    • Up 23 days from 2015
    • Out of 261 work days in 2016
  • Out of these 61 days are in workshops and the rest consulting
  • 18 workshops
    • Up from 14 in 2015
    • Go workshops
    • First docker workshop (teaching with ops specialist)
    • First open enrollment class
    • In Israel, UK, and Poland
      • Only 4 out of the country (vs 8 last year, which is good - I prefer to fly less)
    • Workshops range from 1 to 4 days
  • Revenue up 10% from 2016
  • Several new clients including GE, AOL, Outbrain, EMC and more
Insights
  • Social network keeps providing almost all the work
    • Doing good work will get you more work
  • Workshops pay way more than consulting
    • However can't work from home in workshops
    • Consulting keeps you updated with latest tech
  • Python and data science are big and in high demand
    • However Go is getting traction
  • Having someone else take care of the "overhead" (billing, contracts ...) is great help.
Last Year's Goals
We've set some goals in 2015, let's see how we did:
  • Keep positioning in Python and Scientific Python area
    • Done
  • Drive more Go projects and workshops
    • Done
  • Works less days, have same revenue at end of year
    • Failed here. Worked more and revenue went up
  • Start some "public classes" where we rent a class and people show up
    • Did only one this year
  • Publish my book
    • Failed here, though I'm close. Probably Q1 this year.
Goals for 2016
  • Work less while keeping same revenue
  • Work more from home
  • Publish my book
  • Publish a video course
  • More open enrollment classes

Monday, December 26, 2016

Automatically Running BigQuery Flows

At a current project we're using Google's BigQuery to crunch some petabyte scale data. We have several SQL scripts that we need to run in specific order. The below script detects the table dependencies and run the SQL scripts in order. As a bonus you can run it with --view and it'll show you the dependency graph.

Friday, November 25, 2016

Using built-in slice for indexing

At one place I consult I saw something like the following code:
This is good and valid Python code, however we can use the slice built-in slice to do the same job.

Also when you're writing your own __getitem__ consider that key might be a slice object.

Blog Archive