AskHN Digest

1. Ask HN: How can I learn about performance optimization?

Top comment by Agentlien

For several years I have worked primarily with performance optimizations in the context of video games (and previously in the context of surgical simulation). This differs subtly from optimization in certain other areas, so I figured I'd add my own perspective to this already excellent comment section.

1. First and foremost: measure early, measure often. It's been said so often and it still needs repeating. In fact, the more you know about performance the easier it can be to fall into the trap of not measuring enough. Measuring will show exactly where you need to focus your efforts. It will also tell you without question whether your work has actually lead to an improvement, and to what degree.

2. The easiest way to make things go faster is to do less work. Use a more efficient algorithm, refactor code to eliminate unnecessary operations, move repeated work outside of loops. There are many flavours, but very often the biggest performance boosts are gained by simply solving the same problem through fewer instructions.

3. Understand the performance characteristics of your system. Is your application CPU bound, GPU compute bound, memory bound? If you don't know this you could make the code ten times as fast without gaining a single ms because the system is still stuck waiting for a memory transfer. On the flip side, if you know your system is busy waiting for memory, perhaps you can move computations to this spot to leverage this free work? This is particularly important in shader optimizations (latency hiding).

4. Solve a different problem! You can very often optimize your program by redefining your problem. Perhaps you are using the optimal algorithm for the problem as defined. But what does the end user really need? Often there are very similar but much easier problems which are equivalent for all practical purposes. Sometimes because the complexity lies in special cases which can be avoided or because there's a cheap approximation which gives sufficient accuracy. This happens especially often in graphics programming where the end goal is often to give an impression that you've calculated something.

2. Ask HN: What underrated open source project deserves more recognition?

Top comment by shp0ngle

Bruno https://www.usebruno.com/

https://github.com/usebruno/bruno

Alternative to postman, that's fully local and syncs to git easily. Unlike hoppskotch and insomnia that are free, but offer paid sync, Bruno just works in git. Unlike others, it doesn't dump JSON that's hard to diff, but has its own easily diffable format. You can share your collections in your existing git, with your existing accounts and PRs.

Also has integrated CLI testing.

He plans to sell GRPC later for money. HTTP and GraphQL API works now. https://www.usebruno.com/pricing

3. Ask HN: Feeling quite disheartened about job search, any advice?

Top comment by itake

When I am in situations like this, I try to focus on the process, not the outcome.

Find joy in applying for new jobs, learning about system design and algorithms, or doing practice interviews.

If you follow the process, you will find that job.

----

On a side note, I would encourage you to not lower your pay or communicate you will accept lower pay, because its a signal to employers that you're desperate, no one wants you, and they too should be cautious to hire you. As one of my old favorite managers once told me, "when I gave you that offer, I wanted you to walk in on your first day smiling"

4. Ask HN: Any advice on navigating this job market or pivoting out of tech? (US)

Top comment by dasil003

The thing missing from this post is any sense of whether you have talent and interest in programming. The closest you come is to say you've "gradually lost interest in [...] full-stack web development", but even there you use the language "only domain I have any real-world experience in". It's pretty clear the problem here is job search anxiety. Hey, I don't blame you, I started my software career in the summer of 2000, so I can relate to the doom and gloom.

The thing to remember is that your success is not primarily determined by macroeconomic trends. Sure some industries are clearly on a downward slope and you wouldn't want to start a career there, but tech is not one of them. Software is still eating the world, and yes AI will change it, code may go away, etc, but ultimately understanding and solving problems in complex systems is going to stay an incredibly valued skill for the foreseeable future.

So, are you any good at programming? If so, and it doesn't feel like pulling teeth, then I think you probably have more upside there. If not, and you want something stable and reliable over time, maybe consider the trades. Whatever you do though, understand all career success is based on being able to solve problems, and have the person with the money recognizing that you can provide an above average solution. The best path to this is get a job, any job, learn to do it exceptionally well, and make sure in your first 10-15 years you are continually learning and growing, and building good relationships with the people you work with. If you keep doing that serendipity will find you. No amount of pre-planning or strategizing is a substitute.

5. Ask HN: What is the, currently, best Programming LLM (copilot) subscription?

Top comment by cheema33

I have been a big user of ChatGPT-4 ever since it came out. I have used in various forms. Through the chatbot, through GitHub Copilot etc. For the past year, I have looked high and low to see an alternative that performed better. Nothing came close. Until a few days ago with Anthropic released Claude 3. For software development work, it is very very impressive. I've been running GPT-4 and Claude 3 Opus side by side for a few days. And I have now decided to cancel all AI subscriptions except Claude 3.

For software development work, no other model comes close.

6. Ask HN: How's the current state of hiring in the LLM field?

Top comment by extr

I'm a DS with ~10 yoe mostly in classical ML and I've been wondering how much/how fast I need to pick up "high level" LLM skills. I've seen the way the winds are blowing wrt to the DS job title and so have been moving towards more of engineering roles. But it seems like every job posting right now for ML engineering positions is asking for "pytorch, DL, LLM" experience, even for companies where getting into the nuts and bolts of inference would not really be a core value prop. Like I really find it hard to believe that financial/insurance companies (my primary domain) are building and fine tuning their own models for chat/RAG applications or whatever (or if they are, should they be?). I haven't really seen huge value add from even the most sophisticated RAG setups yet.

I use LLMs every day for code assistance so I'm not totally naive to the whole thing, but I seriously wonder what direction to take my career, exactly. Should I be going all-in on learning CUDA? Is knowing how to wire up a few API calls enough for "soft" applications? Should I in fact be keeping up with the latest RAG techniques (which seem to change every time a new foundational model gets released)? Should I just stay out of the whole thing and double down on classical ML applications in my domain?

7. Ask HN: What laptop are you using to daily drive Linux?

Top comment by aquova

I have a Framework laptop (the original 11th gen Intel one) with Arch Linux and some experiments with NixOS. I've been really impressed with the laptop and their Linux support has been superb. The one caveat is that the battery life isn't great, but I hear that the newer motherboards (particularly the AMD one) greatly improves this. It's nice that I have the option to just drop in a newer motherboard, but I honestly haven't felt that it's aged nearly enough to do that yet.

8. Ask HN: Is anyone considering cancelling their ChatGPT subscription?

Top comment by eveb

I paid for a subscription today and cancelled the subscription today and was refunded. It was freezing non stop, it kept adding color to my black and white line art and then told me it had no control over that, then it was telling me I ran a single prompt too many times. I said it kept getting it wrong that's why and it would do it again but it was slowing me down every time it was refusing my commands. The final straw was when it shut down and said I'd used too many requests and to come back near midnight the next day... so I'd waste a whole day of not being able to use it because I'd be in bed by midnight. If I'm paying, I want to able to use the thing for more than 20 images and I don't want to have to argue with the chatbot about what I'm doing. It should have no say or thoughts about how many times I've run a prompt.

9. Ask HN: What AI assistants are already bundled for Linux?

Top comment by yjftsjthsd-h

NixOS has packages for llama-cpp and ollama: https://search.nixos.org/packages?channel=23.11&from=0&size=...

Edit: And Arch packages ollama officially - https://archlinux.org/packages/?sort=&q=llama&maintainer=&fl... - and a few things in the AUR - https://aur.archlinux.org/packages?O=0&K=llama

10. Ask HN: Code Llama 70B on a dedicated server

Top comment by paulzain

I run CodeLlama70B in my home lab using two RTX 4090s and the MLC AI framework. It provides Muti-GPU support and is extremely fast on consumer grade hardware. I'm seeing about 30 tokens/sec.

https://blog.mlc.ai/2023/10/19/Scalable-Language-Model-Infer...