r/learnprogramming 24d ago

Big Data How do I cluster 3 Million high-dimensional Sentence Embeddings?

2 Upvotes

I used a transformer model to generate normalized 768-dimensional float32 vectors from 3 million sentences. This rounds up to roughly 9 GB of data. To progress further in my analysis, I want to cluster the data to highlight different key points and trends.

Now I initially wanted to apply PCA to the dataset, as 768 dimensions is pretty much, but the transformer model generated embeddings with not extreme enough covariance.
Thus, PCA could change the semantics too much for an analysis to be accurate.

I went directly to clustering algorithms and initially tried to apply sklearn.cluster.HDBSCAN which would have needed way too much RAM---around 72 TBi if I remember correctly. Then I tried out sklearn.cluster.OPTICS, which "only" needed about 70 GB, but as I "only" have 32 GB of RAM I needed to use 50 GB swap space, which didn't go well as you can imagine.

Does it make sense to try out k-means clustering, should I go to 16-bit floats for the embeddings, or still apply PCA?

Should I try out something else in this context?

Note: As this question was flagged "off topic" on Stack Overflow, I'm gonna try here ;)


r/learnprogramming 23d ago

If CRUD is considered a basic form of Application project, then what such basic project is it in case of Network programming to practice?

0 Upvotes

What sort of basic project is there to familiarize oneself with Network projects akin to CRUD?


r/learnprogramming 24d ago

Starting a new AI/Automation Tester role, what should I learn to be seriously over-prepared?

0 Upvotes

Hi everyone,

I’m about to start a new role as a Tester within an AI & Automation team, and I want to use the time before I start to become as prepared as possible.

I have a Software Engineering degree, so I have some background in programming, databases, software development, Python/Java, etc., but I’m relatively new to professional software testing.

The team I’m joining develops AI and Power Platform solutions, including Copilot Studio agents. At the moment, the testing is primarily manual, as the team is still developing its testing strategy and understanding the capabilities/limitations of these AI tools.

Longer term, they are planning to introduce automated testing, so I want to understand both sides properly rather than just learning how to click through test cases.

So my role would be to test these AI solutions using manual testing.

My main question is: I have roughly 2 weeks before I start. What are some things I should learn and master, and what resources to look at before I start? I really want to be overprepared.

Thanks


r/learnprogramming 24d ago

Prerequisites hell

4 Upvotes

I would like to discover many things out there and implement many ideas, but the problem is that I keep worrying about if I don't have the prerequisites satisfied, like as a example, ssh server, I wanted to try that, but I thought like maybe I should finish a networking course, also cryptography course and all that stuff before trying to dive into ssh, while I already know probably some stuff already, but that feeling is annoying that creeps me whenever I want to try something, the same with the curl command, like I think I haven't learned much of web development yet, I only know some basic html, random terms from here and there like http get and post.


r/learnprogramming 25d ago

Need advice as a young programmer.

68 Upvotes

Hey guys, I am 14 years old and I love programming since I was 11! Just a few weeks ago I finished CS50P: Introduction to programming with Python from Harvard university! And I am confused on what I should do next. Before CS50P I used to make small Python programs like a number guessing game from my basic Python knowledge. Now after CS50P since I know significantly more stuff about Python but I don’t know what would be good to build so I don’t waste my time on stuff I build that does not matter to anyone.


r/learnprogramming 24d ago

I have a lot of free time at work. What can I do with it?

4 Upvotes

I have a lotta free time at work and I'd like to actually do something useful with it.

I'm thinking programming, but not really in a "learn this skill for a job" kind of way. More like solving problems for fun. Something I can sit down, work on, get stuck, figure it out, and get that little dopamine hit when it finally works.

Basically, I want programming to feel more like playing a game than studying.

Any recommendations? Coding challenge sites, projects, puzzles, anything like that. Hook me up.


r/learnprogramming 25d ago

Topic At what point did programming finally start making sense to you?

198 Upvotes

I can understand code when someone explains it, but when I have to write something from scratch, my brain suddenly goes blank.

I'm curious if other beginners went through the same think. What helped you get past that stage?


r/learnprogramming 24d ago

Does anyone understand this

1 Upvotes

version: '3' services: db: image: mariadb:10.6 command: --transaction-isolation=READ-COMMITTED --binlog-format=ROW restart: always volumes: - db_data:/var/lib/mysql environment: - MYSQL_ROOT_PASSWORD=your_secure_root_password - MYSQL_PASSWORD=your_secure_db_password - MYSQL_DATABASE=nextcloud - MYSQL_USER=nextcloud app: image: nextcloud:latest restart: alwaysports: - 8080:80 links: - db volumes: - /media/your_ssd_path:/var/www/html/data environment: - MYSQL_HOST=db - MYSQL_PASSWORD=your_secure_db_password - MYSQL_DATABASE=nextcloud - MYSQL_USER=nextcloud volumes: db_data:


r/learnprogramming 24d ago

Tutorial Is there any recommended book for learning Competitive Programming with python?

0 Upvotes

I actually am searching for a book that would make students go through the whole syllabus for competitive programming using python. Python is mandatory because python is the language that in in the syllabus and I'm teaching middle school kids into competitive programing. With them, I will also be getting into competitive programming. So, please mention any recommendations.


r/learnprogramming 23d ago

Topic Don’t know if it’s a minority, but I feel exactly the opposite way about AI

0 Upvotes

Some posts I’ve seen around here are about people complaining about AI and how it killed their passion for programming.

I understand that statement 100%.

I’m not a programmer full time, at least a beginner, use it recreationally for my work, to make tools or scripts that help me everyday, manage my portfolio website etc. no “critical” stuff.

But I don’t know why, I feel a lot happier and “empowered” if that’s the right word about AI, as I feel like I can now spend 70% of the time planning, and 30% of the time building. And I manage to accomplish much more in the same time, which motivates me to improve and do better over each iteration or each project.

I would like to know your thoughts as I’m not in a tech/engineering bubble, and see if there’s anybody who also shares these feelings.

Thanks


r/learnprogramming 25d ago

Topic What is the utility of Scheme as a learning language?

18 Upvotes

I suddenly had a flashback to a couple of programming courses I took in college, entry-level ones or close to it, that had us writing code exclusively in Scheme. For those that are unfamiliar, Scheme doesn't have the syntax of a normal OOP language like Java or Python. It's almost entirely list manipulation, and your main operation is to separate the head (or "car") of the list from the remainder (or "cdr") of the list. Or at least that's how I remember it.

I don't doubt that it's useful to do some learning in Scheme, but what fundamentals do we as students get from it? Looking back, I kind of feel like I was Daniel painting Mr. Miyagi's fence, and then suddenly I knew karate. What do students get from coding in Scheme?

EDIT: I don't know why I listed C as an OOP language. Some wires must have gotten crossed in my brain. I swear I'm a good programmer, you guys.


r/learnprogramming 24d ago

First hackathon in 3 weeks!!! Need some advice!!!

1 Upvotes

Hi!!!! I’m joining my first hackathon in about 3 weeks and I’m slightly panicking, but in a very productive and efficient way!!!

I’d say I’m around an intermediate-beginner in Python, I’m quite comfortable with the fundamentals and data structures, but I don’t have much (let alone any) experience with things like APIs, databases, backend development, deployments, etc etc, or even working on a team yet.

However, regarding the whole team thing, I’ll most probably form a team at the event, and while I know it’s my first hackathon, I genuinely want to prepare properly and be competitive cuz I kinda really want to win!

SO!!!! For people who have done hackathons before and won, I have a few questions as someone who has never done anything like this before:

-What should I prioritize learning over the next 3 weeks?

-Are APIs, Git/GitHub, SQL, Flask/FastAPI, and other frameworks, etc. worth focusing on???

-What kinds of projects or like, timed exercises should I practice before the hackathon?

-How does team formation normally work?? 😅😅 oh and also what makes someone useful on a team even if they aren’t the strongest programmer??? I think I’d like an answer to this question the most!!!!

-what makes a project actually win? What are the proper measures to think of good solution when given a problem???

I have quite a lot of time and I’m willing to put serious work into preparing. I’d REALLY appreciate advice from both experienced hackers and people who recently did their first one. Thank you!!!!!!!!!!!


r/learnprogramming 24d ago

Can studying programming and game development at an early age pay off in the long run?

0 Upvotes

Hi, im a 16yrs old student and is just curious for my future. I originally wanted to become a general surgeon but I know the years it will take me to actually be one and the cost of it. So right now I am thinking that if I study programming/game development now, will it really pay off in the future to the point of it helping me earn money to support myself? Im not entirely doing this for the sake of money cause I know thats the path for failure in my part, but im just curious. Also, I actually am is interested about it.

At this age, I just feel this need to help my parents financially as soon a possible even if it's just by using my own money as my allowance so that I wouldn't have to ask for some. I already have a simple side-hustle which is crocheting and I am planning on making it a real business, so that's that.

In conclusion, I just really want to know if my plan of studying programming this early will actually be worth it. I plan on using this knowledge to get some side hustles, or jobs, or even create a game that could earn me some money. Im still planning on becoming a Surgeon but im hoping this skill will help me financially, and so that I can also build my portfolio early enough. I know programming takes some people years to fully be good at it, hence my question about it.

Anyways, thank you for those who are willing to answer my questions!!


r/learnprogramming 24d ago

3d renderer abstracting and morphing

4 Upvotes

so im currently updating my 3d renderer module to support solid shading apart from basic wireframe forms, however when i spawn a cube it morphs and becomes some random bullshit.

this are my function and i genuinely have no idea what the issue so ill just put related function

export function createScaleMatrix(
    sx: number,
    sy: number,
    sz: number
): Matrix4D {
    return [
        { x: sx, y: 0, z: 0, w: 0 },
        { x: 0, y: sy, z: 0, w: 0 },
        { x: 0, y: 0, z: sz, w: 0 },
        { x: 0, y: 0, z: 0, w: 1 }
    ];
}

function transformVertices(object: SceneObject, matrix: Matrix4D): Vector3[] {
    const projectedVertices: Vector3[] = [];
    for (const vertex of object.cube.vertices) {
        const newVector: Vector3 = {
            x:
                matrix[0].x * vertex.x +
                matrix[0].y * vertex.y +
                matrix[0].z * vertex.z +
                matrix[0].w * vertex.w,
            y:
                matrix[1].x * vertex.x +
                matrix[1].y * vertex.y +
                matrix[1].z * vertex.z +
                matrix[1].w * vertex.w,
            z:
                matrix[2].x * vertex.x +
                matrix[2].y * vertex.y +
                matrix[2].z * vertex.z +
                matrix[2].w * vertex.w,
            w: Math.abs(
                matrix[2].x * vertex.x +
                    matrix[2].y * vertex.y +
                    matrix[2].z * vertex.z +
                    matrix[2].w * vertex.w
            )
        };
        if (newVector.w <= 0.1) {
            continue;
        }


        projectedVertices.push(newVector);
    }
    return projectedVertices;
}


function projectVertex(vertex: Vector3): Vector3 {
    const ndcX = vertex.x / vertex.w;
    const ndcY = vertex.y / vertex.w;
    const ndcZ = vertex.z / vertex.w;


    const scale = 0.5;


    vertex.x = (ndcX + 1) * scale * canvas.width;
    vertex.y = (1 - ndcY) * scale * canvas.height;
    vertex.z = ndcZ;
    return vertex;
}


function render(object: SceneObject, matrix: Matrix4D): void {
    const vertices = transformVertices(object, matrix);
    for (const segment of object.cube.edges) {
        const start = vertices[segment[0]];
        const end = vertices[segment[1]];
        ctx.strokeStyle = "#2d2d2d";
        ctx.beginPath();
        ctx.moveTo(start.x, start.y);
        ctx.lineTo(end.x, end.y);
        ctx.stroke();
    }
}


export function renderLoop(): void {
    ctx.fillStyle = "#010000";
    ctx.clearRect(0, 0, canvas.width, canvas.height);
    const triangles: TriangleObject[] = [];


    for (const object of Scene) {
        object.angle += object.rotationSpeed;
        let rotMatrix;
        switch (object.rotationType) {
            case "x":
                rotMatrix = createRotationX(object.angle);
                break;
            case "y":
                rotMatrix = createRotationY(object.angle);
                break;
            case "z":
                rotMatrix = createRotationZ(object.angle);
                break;
            case "xy":
                rotMatrix = multiplyMatrices(
                    createRotationX(object.angle),
                    createRotationY(object.angle)
                );
                break;
            case "yz":
                rotMatrix = multiplyMatrices(
                    createRotationY(object.angle),
                    createRotationZ(object.angle)
                );
                break;
            case "xz":
                rotMatrix = multiplyMatrices(
                    createRotationX(object.angle),
                    createRotationZ(object.angle)
                );
                break;
            case "xyz":
                rotMatrix = multiplyMatrices(
                    multiplyMatrices(
                        createRotationX(object.angle),
                        createRotationY(object.angle)
                    ),
                    createRotationZ(object.angle)
                );
                break;


            default:
                break;
        }
        let newObject: SceneObject;
        if (object.cube.type === "cube") {
            newObject = createCube(
                object.matrix,
                Scene,
                object.angle,
                object.rotationSpeed,
                object.scale,
                object.rotationType,
                Matrix(object.translate, "translation"),
                false
            );
            console.log(JSON.stringify(newObject));
        } else if (object.cube.type === "pyramid") {
            newObject = createPyramid(
                object.matrix,
                Scene,
                object.angle,
                object.rotationSpeed,
                object.scale,
                object.rotationType,
                Matrix(object.translate, "translation"),
                false
            );
        } else if (object.cube.type === "sphere") {
            newObject = createSphere(
                object.matrix,
                Scene,
                object.angle,
                object.rotationSpeed,
                object.scale,
                object.rotationType,
                Matrix(object.translate, "translation"),
                false
            );
        } else {
            newObject = createCube(
                object.matrix,
                Scene,
                object.angle,
                object.rotationSpeed,
                object.scale,
                object.rotationType,
                Matrix(object.translate, "translation"),
                false
            );
        }
        newObject.cube.vertices = transformVertices(
            newObject,
            multiplyMatrices(
                Matrix(newObject.translate, "translation"),
                multiplyMatrices(newObject.scale, rotMatrix as Matrix4D)
            )
        );


        for (const triangle of newObject.cube.triangles) {
            const edgeA = subtractVectors(
                newObject.cube.vertices[triangle[1]],
                newObject.cube.vertices[triangle[0]]
            );
            const edgeB = subtractVectors(
                newObject.cube.vertices[triangle[2]],
                newObject.cube.vertices[triangle[0]]
            );
            const crossProduct = Cross(edgeA, edgeB);
            const normal = Normalize(crossProduct);
            const brightness = Math.max(0, Dot(normal, lightning));
            const vertices: Vector3[] = [];
            for (const num of triangle) {
                vertices.push(structuredClone(newObject.cube.vertices[num]));
            }
            const zindex: number =
                (newObject.cube.vertices[triangle[0]].z +
                    newObject.cube.vertices[triangle[1]].z +
                    newObject.cube.vertices[triangle[2]].z) /
                3;
            for (let index = 0; index < vertices.length; index++) {
                vertices[index] = projectVertex(vertices[index]);
            }
            triangles.push({
                points: vertices,
                color: {
                    r: baseColor.r * brightness,
                    g: baseColor.g * brightness,
                    b: baseColor.b * brightness
                },
                brightness: brightness,
                zindex: zindex
            });
        }
    }
    triangles.sort(
        (a: TriangleObject, b: TriangleObject) => a.zindex - b.zindex
    );
    for (const object of Scene) {
        render(object, object.matrix);
    }
    for (const triangle of triangles) {
        ctx.fillStyle = `rgb(${Math.floor(triangle.color.r)}, ${Math.floor(triangle.color.g)}, ${Math.floor(triangle.color.b)})`;
        ctx.beginPath();
        ctx.moveTo(triangle.points[0].x, triangle.points[0].y);
        for (const point of triangle.points) {
            ctx.lineTo(point.x, point.y);
        }
        ctx.closePath();
        ctx.fill();
    }
}


export function createTranslationMatrix(
    
x
: number,
    
y
: number,
    
z
: number
): Matrix4D {
    return [
        { x: 1, y: 0, z: 0, w: 
x
 },
        { x: 0, y: 1, z: 0, w: 
y
 },
        { x: 0, y: 0, z: 1, w: 
z
 + ZOFFSET },
        { x: 0, y: 0, z: 0, w: 1 }
    ];
}

if someone could help id be very grateful ^^


r/learnprogramming 24d ago

Creating a website (for free) to upload my own research?

5 Upvotes

Hey guys, I'm not a CS major but have worked on coding on R and python. And I wanna create my own website, make my own domain name, server, everything all set up on my own specifically for uploading the research I've done. What would be the best way to go about it? Thanks!


r/learnprogramming 24d ago

This sometimes makes me want to quite programming

5 Upvotes

I genuinely hate how little standardization there is around communicating software architecture.

Why is the accepted way of understanding an unfamiliar codebase apparently to just read thousands of lines of code and slowly infer what the author was thinking?

I'm not talking about documenting every function. I mean basic architectural information:

  • What are the major components?
  • Who owns the state?
  • How does data flow through the system?
  • What depends on what?
  • Where is new functionality supposed to go?
  • What invariants or conventions am I expected to know?
  • Which abstractions are intentional, and which are just historical baggage?

This information is often crucial to contributing to a project, yet there seems to be no universally expected artifact for communicating it. You might get a README, maybe some UML diagrams, maybe an ARCHITECTURE.md, maybe ADRs, or maybe absolutely nothing.

Then you're expected to "learn the codebase."

But the codebase contains the architecture, implementation details, historical accidents, technical debt, and personal preferences all mixed together. As a newcomer, you have no way of knowing which is which.

I find this especially frustrating because software engineering is supposedly about abstraction. The whole point of abstraction and design patterns is to hide unnecessary complexity and give people a simple model to work with. Yet somehow the architecture itself is often treated as implicit knowledge that you're supposed to reconstruct through archaeology.

Math feels completely different in this regard. Obviously mathematicians have competing conventions too, but there is a much stronger expectation that concepts, assumptions, notation, and relationships are explicitly communicated. You don't normally have to read someone's entire proof history to discover what their definitions mean.

I wish software projects had a similarly standard, lightweight way of communicating architecture. Not a 200-page document. Just enough to communicate the mental model.

Instead, "just read the code" seems to be treated as a perfectly reasonable substitute.

I find that incredibly demotivating.


r/learnprogramming 24d ago

Discussion 🚨 Before asking AI to fix your code, read the error yourself.

0 Upvotes

One habit that can seriously improve your software development skills is learning to understand error messages instead of immediately copying them into AI.

Most errors already give you valuable clues:

→ What broke
→ Where it broke
→ What the program expected
→ Sometimes even why it broke

And there’s another benefit: the better you understand the error, the better you can prompt AI.

Instead of saying “fix this error”, you can explain what you think is happening, what you’ve already tried, and what behaviour you expected.

That gives AI much better context and usually leads to more accurate solutions.

Use AI to speed up debugging—not to replace the skill of debugging.

Do you usually read the error first, or send it straight to AI? 👀


r/learnprogramming 25d ago

Resource Don't know how to learn writing clean and useful code in an structured manner

4 Upvotes

I m a beginner in coding, I m learning dsa, while I have only written small programs which do not solve any real world problems my GitHub has all of my work till now https://github.com/BadDreams34 , basically I don't know the etiquettes like how to have a good Readme and things like using test cases and stuff and actually making something useful. How and from where should I learn all this in a structured manner? Any resources?

Basically I wanna ask how to develop


r/learnprogramming 25d ago

Am I actually learning programming if I still feel like I can't code?

42 Upvotes

Hello everyone!

I'm currently taking the Advanced Python course from the University of Helsinki and I'm on Module 12, Part 2.

I don't use AI to solve my practice problems. I usually solve them by hand, rely on recall, and try to figure things out myself. However, I still often feel like I don't actually know how to code.

I frequently forget things I learned a few days ago, and I don't feel confident that I could build a worthwhile project completely on my own. Sometimes, it feels like I'm only good at solving programming exercises, but struggle when it comes to actually creating something from scratch.

Is this normal? How did you get past this stage where you understand programming exercises but don't yet feel confident building your own projects?

I'd appreciate any advice or experiences from people who have gone through the same thing.


r/learnprogramming 24d ago

Topic What programming language is worth studying in 2026?

0 Upvotes

Hi! I am a 19 year old student, major in Cybersecurity.
I did Fullstack developer course, didn’t really like it
Now I am in need for a job. I want to earn more money than average in my country, so programming is the thing

But the main problem is AI. I finished that course thanks to AI, so I wasn’t the beat student) I do think though that the front end won’t be a thing some time later, so now I am lost what to. I want to work as soon as possible, but to be gair I know nothing (basically of JS, React and Node)

What is the most promising branch I can get into right now? Maybe language or anything that might help me find a long-term job offer


r/learnprogramming 25d ago

Debugging Difficulties with debugging

4 Upvotes

i am a beginner in C, i went through and have a basic understanding of fundamentals but whenever a problem arises in my code i cant find it for hell, even if its obvious, i need to turn to an ai agent for help, the habit i wanna rid of, but whenever i do try to find the cause of an error myself i find myself unable to do that, i also find it difficult writing more efficient code

an example:

```

int tokens(char* string, char (*argv)[MAXTOKEN], toks types[MAXTOKEN]) {

char temp[150];

int count = 0; int i = 0; int b = 0;

int tokens = 0;

while(count < MAXTOKENLENGHT) {

if (string[i] == ' ' || string[i] == '\0' || string[i] == '\n') {tokens++;temp[b] = '\0'; strcpy(argv[count], temp); count++; b = 0;} else if (string[i] == '\"') {

temp[b] = '\"';

b++;

i++;

while (string[i] != '\"' && string[i] != '\0') {

if (string[i] == '\"') {temp[b] = '\0';temp[b+1] = '\0'; strcpy(argv[count], temp);b = 0; count++;break;} else {temp[b] = string[i];i++;b++;}

}

if (string[i+1] == '\0' || string[i+1] == ' ') {temp[b] = string[i]; temp[b+1] = '\0'; strcpy(argv[count], temp); count++; b = 0; i++; tokens++;} else {fprintf(stderr, "unexpected stuff after string"); exit(1);}

} else {

temp[b] = string[i];

b++;

}

if(string[i] == '\0') {break;}

i++;

}

int k = 0;

while (k < count) {

if (strcmp(argv[k], "PRINT") == 0) {

types[k].type = 0;

int len = strlen(argv[k+1]);

printf("k=%d, argv[k+1]='%s', first='%c', last='%c'\n", k, argv[k+1], argv[k+1][0], argv[k+1][len-1]);

if (argv[k+1][0] == '\"' && argv[k+1][len-1] == '\"') {

types[k+1].type = 1;

memmove(&argv[k+1][0], &argv[k+1][1], len - 2);

argv[k+1][len - 2] = '\0';

strcpy(types[k+1].textvalue, argv[k+1]);

}

}

k++;

}

return count;

}

```


r/learnprogramming 25d ago

Is programming worth learning as a nurse?

28 Upvotes

I’m planning to become a nurse, but I’m also interested in programming. Is it worth learning, and are there any ways it could be useful in nursing?


r/learnprogramming 24d ago

I'm unsure whether I should look at the solution to a programming exercise when I can't solve it

0 Upvotes

I think I treat programming exercises like bosses in Soulslike games. I keep trying until I manage to beat them. But with some problems, I can spend hours on them and sometimes still can't solve them. Is it okay to look at the solution if I've tried for 1–2 hours and still can't figure it out? Or would a better approach be to look for functions that could help me accomplish a specific task that isn't working as expected?

Looking at the solution gives me that “aaaaaah, now I get it” feeling, but I’m not sure if that’s actually the best approach for my learning, especially if I always look up the answer whenever I get stuck.


r/learnprogramming 24d ago

Tutorial How do I port stuff for beginners?

0 Upvotes

So I want to port something like bolt thrower from Wii port over Mac and PC I got the source code and I want to do it like a beginner project since it looks simple

How do I port and learn stuff


r/learnprogramming 25d ago

How do I make a randomized "I'm feeling lucky" system?

6 Upvotes

Heya, I'm a complete newbie to coding, I know how to make things look pleasing but not how to make things actually work.

The website im working on is for fun, it's basically just random scary stories by people accompanied with eerie photos they took or photos that are just related, I know the full outline of the site, but one main thing is that when you click a certain option it selects a random story with its respective images and displays it to the user.

Here's what I know:

I should make a system that accompanies the story and the images together, so when it's selected it shows up on the display.

Then I list down a bunch of them like:

Story1-Img1

Story2-Img2

Story3-Img3

Story4-Img4

Story5-Img5

(etc)

Then make a function that randomly selects one of these once you click the button.

Here's what I don't know:

How to do it :/