“A Cold War Every Day” inside Apple's internal tools group

ttepasse | 334 points

Sounds like a bank, a government agency, or any business over a certain size.

I tell people you only call it politics when you are losing. More accurately, it's a layer of literal stupidity above the competent to shield the money side of the company from the leverage that operations people would have if they had any information about how the money side worked.

Instead of a hierarchy, rethink a company as a hub and spoke model with concentric rings. The main differences are the implication in a hierarchy that there is "gravity," keeping people down and that they need energy and leverage to climb "up," which further implies there is a place to "fall," and that there is only one way "up," instead of many possible paths to the centre from all directions. There is no gravity, only gates and barriers, and even these are just information. Politics is how a middle manager runs interference and creates distractions to make sure you can't see over, around, or through them, and that the people behind them closer to the money can't see you. Tech is usually outside the main perimeter, mediated by contracting companies or middle managers whose job is to compartmentalize the value people create, and be sure it is replaceable.

Viewed this way, of course this demented political farce is how Apple works, because it's how everything seems to work when you have internalized the precise and specific mental model someone uses to take advantage of you.

Sorry if you can't unsee it now, but hopefully it will be funny and we can get good, competent people who value tangible skills into positions of power.

motohagiography | 4 years ago

Fascinating to see an article about IS&T, definitely not something I ever expected to hear about on HN!

I was part of a team inside Apple that developed internal-facing tools for the retail teams. Our team was formed because of the extremely high cost in dollars and time that IS&T wanted to charge to develop some fairly straightforward tools. My team was taken from the departments that used the tools, and we developed things that were very custom-fit to what those departments needed.

Eventually politics overcame us, and IS&T finally managed to take over our team. We all became contractors in order to support the migration to their infrastructure, and continue development of the tools. A bit later we all became fired as our project was outsourced to contractors in India, managed by IS&T PMs who had no idea what our tool was used for and had never been inside a retail store. The tools all died shortly after that, some killed by IS&T, some petered out due to lack of use now that they no longer worked correctly or were a good fit for the users.

As far as we could tell, IS&T was run as a unique company inside Apple, which did it's fair share of price gouging in order to make itself money to keep going. Its purpose never appeared to be helping Apple customers or Apple employees, it was simply to get bigger, absorbing more money and power wherever possible, with no apparent reigning in from the parent org.

Although my experience is several years old, everything in this article rings true. The contracting companies they had us working for were taking a huge cut, the quality of the code they produced was dismal, (as soon as we were no longer allowed to re-write their code major things began breaking almost immediately) and people getting transferred around constantly and having no time to understand any one project was common. (rkho's comment about their hiring process seeming like it was simply a beard for a nepotistic contractor conversion was something we definitely saw a number of times.)

All in all it was an extremely eye-opening experience. Considering how "do it the Apple way" every other department we interacted with was, being in the IS&T buildings was like landing on an alien planet.

IST-Throwaway | 4 years ago

A while ago I interviewed for a senior fullstack role with a very homogenous team at Apple -- every single person on the team had been converted from the same consulting company (I believe it was Wipro).

It was a bizarre interview with three of the engineers on the team on the same call asking me very specific programming language trivia and nothing about my experience nor ability to navigate complex political structures -- you know, the things that you would expect a senior fullstack engineer to have before advancing them to the next round.

Unsurprisingly, I wasn't advanced to the next round. In light of this article, I can't help but wonder if it was all a ruse to simply demand another nepotistic FTE conversion from their same consultancy.

rkho | 4 years ago

I had a similar experience w/Apple. A startup I worked for was brought into Cupertino for a meeting w/their internal business teams. They wanted us to build them an app which seemed relatively simple involving their internal Cafe Mac cafeterias, data centers, and possibly retail locations. It was something that a company like Apple could build in their sleep. But the business folks we met with said that's how it is at Apple, all the engineering talent goes towards the product side and almost nothing is left for internal IT. They told us how they struggled to get anything done and there were almost no resources available, so the business teams had to go and hire their own IT if they needed things done.

coleca | 4 years ago

I could use some blog posts about quite functional teams.

Like someone on a team for a decade talking about why it works so well and discusses cultural and political issues and how the team overcame them.

Waterluvian | 4 years ago

> For Apple, fixing its broken IS&T division would not only be the right thing to do from a moral standpoint — it would help the company’s business as well. If Apple is going to become inventive again, it will need to give its employees more time to develop new ideas.

I actually laughed. What kinds of applications do people think IS&T is building? Supply chain management, HR, support software (Apple forums anyone?)... I don't see how Apple sticking its core engineering resources or at all innovating within IS&T helps their business.

Steve was the one that coined the 10X rule, or popularized it. That idea must still hold within Apple's executive ranks. I guarantee Apple's top engineers never interface with IS&T and their time won't be saved by improving it. In fact I would bet that managing the deliverable of one of IS&T's projects is a sign that you're not one of the golden ones.

That said, I obviously think these contractors should be treated much better and overworking people for low pay is never okay. Maybe Apple could use some smaller dev shops that treat their people better.

epberry | 4 years ago

"They’re just fighting for the roles,” Sabapathy told me. “That’s all they care about, not the work, not the deliverables, the effort they put in, or even talent. They’re not looking for any of those aspects.”

This is spot on for every large organization I ever consulted for. Moreover. Even in much smaller businesses, typically starting at around 70-100 people (sooner if management has prior large business experience) you can already see this pattern starting to seep in.

This is also why the whole private vs public sector efficiency debate is farcical. They are both identical in this aspect.

PeterStuer | 4 years ago

Sounds like the politics of every large corporate IT department, but with some twists given that it's Apple. Maybe this helps explain why Apple is not Amazon.

riskneutral | 4 years ago

Is there any big consumer product company out there that doesn't deprioritize management and engineering talent for internal tools and systems ?

seemslegit | 4 years ago

IS&T is run by the CFO and it shows.

jldugger | 4 years ago

The contract companies (Wipro, Infosys) hire graduates by the 100's from mediocre engineering colleges, slap on a little bit of training and advertise them as world class. The starting salaries are something like INR 25,000 (USD 350) per month. An onshore posting for these low skilled graduates is a quick way to make money. So they suck up to their managers who suck up to bigger managers. Their main criteria is to get hold of contract work by undercutting, knowing very well there is vendor lock in.

samdung | 4 years ago

Body shop wars. What gets really interesting is working for a body shop with consultants from 2 other shops on the same team.

5cott0 | 4 years ago

IS&T shouldn’t be part of Apple. It’s culture and engineering talent aren’t representative of core engineering.

captainredbeard | 4 years ago

"Don't work in IS&T" has been the common wisdom in Blind for years now. Interesting how long it takes for the media to pick up stories like this.

ummonk | 4 years ago

Is this the team handling iTunesConnect and other such "services"? I'd definitely believe it.

larrik | 4 years ago

The part I don't get is, why does a company, especially one like Apple who would count software quality as a core competency, choose to pay $120-150 an hour to get contractors who aren't even good enough to command a salary (or hourly rate) of half that. I get why like, United Airlines does that. Or the US Government. They probably rightly assume that they can't properly manage a huge software development or IT org. But Apple? Really? What the hell are they gaining? If they paid a developer full time $120k a year plus benefits that still beats $120+ an hour. WTF.

xp84 | 4 years ago

The days when the primary mission of IS&T was to develop custom software to run Apple are probably long past. The big exception is Radar, an ideosyncratic issue-tracking system which was developed in-house by IS&T around 1990. As far as I know, Radar is still used in R&D and elsewhere at Apple.

Tim Cook probably couldn't care less about the internal affairs of IS&T. As long as they do their job, play by the rules, and don't impose too much overhead on the company, it's all good.

chmaynard | 4 years ago

I would be grateful if any Apple insiders here can discuss the risk of layoffs if one were to join the company now.

I have an offer to join an engineering team at Apple, but the chances of getting hit by layoffs is giving me pause. More details on my situation are here: https://news.ycombinator.com/item?id=22809769

apple2019 | 4 years ago

Sounds exactly like government contracting.

danzig13 | 4 years ago

Lol. Just like Govt contracting.

rclayton | 4 years ago

Why are you afraid to mention the word "Indian"?

throwaway321213 | 4 years ago

Does Radar belongs to IS&T?

And I wonder if this IS&T power grabbing and politics happen and grow within the past 10 years. i.e Happen under Tim Cook's watch.

It surely doesn't seems very Steve Jobs approach.

ksec | 4 years ago

One of the world's most successful and wealthy companies in history shouldn't be bottom fishing for the best "deal" on employees. I wonder how much better they could be if they decided to hire at the top of the market like Netflix instead of the bottom.

That is probably why Apple software is among the worst in the industry, and it's a shame. I know IS&T isn't strictly software like iTunes but it fits the pattern I've seen at Apple. But I guess if it doesn't affect their sales maybe I'm wrong and it doesn't matter and they are right.

ping_pong | 4 years ago