How Fair are Facebook Hacker Cup's Rules?

Facebook Hacker Cup is different from many other competitive programming competitions due to its submission format: instead of automatically judging competitiors’ submissions via an online judge, competitors are given the input file and six minutes to generate the answer for that input file locally. This difference in submission format is very peculiar and allows for unorthodox methods of solving problems. For example, you could use MATLAB or Wolfram Mathematica. You could use parallel processing across multiple devices (which isn’t just theorycrafting, it’s actually doable as shown in this recent blog). All of this can make the competition more interesting, but it also begs the question of fairness.

The thing is, when you have to produce the output locally, your hardware plays a significantly larger role in your running time. What made online judging fair was that everyone’s submission ran in the same environment under the same conditions (or as close as possible, since some online judges use multiple machines for judging). And for anyone wondering, there certainly is a difference in hardware performance, even just between a laptop vs a PC. I can personally attest that when I used to use my laptop, I was experiencing almost 5 second compile times when using #include <bits/stdc++.h>, reduced slightly by precompiling headers. After getting a PC, my compile times even without precompiling headers was easily under 1 second, sometimes 2. Actual execution times also received similar boosts. And I can only imagine the difference if using a more powerful machine or multiple. When I interned at a machine learning lab last year, we used supercomputers with GTX Titan Blacks for faster training of the neural networks, and I can only imagine applying such computational power to FHC.

Most of the time, this difference in hardware won’t matter if you come up with the intended solution. FHC problem bounds are small enough that if you code a solution with correct complexity, you can expect to finish running on the input under a minute even with the crappiest hardware. Where it starts matter is when you have less optimal complexities, like $\mathcal O(n^2)$ for $n \leq 10^5$. I feel like cheesing with high computational power is definitely a viable strategy, just that no one has opted to do so yet because FHC is primarily targeted towards competitive programmers who focus more on solving problems with thinking. And if this strategy ends up being viable, these contests can quickly devolve into being pay-to-win instead of being based on algorithm solving prowess.

Regardless, I don’t think this is a serious threat to the competition at the moment. As I mentioned before, FHC is mainly targeted towards competitive programmers, not machine learning scientists with access to insane cloud computing from their lab. It’s just a thought I had. In the worst case, imagine if FHC problems started adapting to account for increased computational power. Problems start having $\mathcal O(n^2)$ as the intended solution for $n \leq 10^5$. That would be funny.


Thanks for reading! I apologize for posting what is essentially a filler article. I’m currently working on a longer tutorial article, but between classes and other responsibilities it’ll take more time to complete, so be on the look out for that.