# Getting into programming work as a physics grad



## HAL (May 10, 2014)

I'm a physics student - 3rd year so almost finished my bachelors. I took a course in Python last year wherein the lecturer told us we should think seriously about it because a lot of physics grads go into programming.

Seems believable, according to this: http://www.hecsu.ac.uk/assets/assets/documents/wdgd_science_data_overview_2015.pdf

I got a 1st in the Python course. I also got a 1st in the C++ course I did the previous year, but it was 'introductory' so barely touched on classes until the last couple of lectures, and I'd be screwed if anyone asks me what polymorphism is.

So now I'm thinking more seriously about focusing on this area and trying to take the programming route after graduation. The problem is I honestly I have no idea at all what I need to focus on, or how even a physics grad with a little bit of programming knowledge breaks into that world.

Personally, I don't think my skills are anywhere near up to scratch. I've uploaded the Python project I did to GitHub so anyone with any knowhow can tell me where I'm at so far. Here it is: https://github.com/Stuart88/Physics_Muon_detector_coursework

I'm going to start some C++ and Python projects just to build up a bit of a portfolio but I don't know what standard these should be either. It would be embarrassing to create a load of pathetic, amateur projects without realising how simple they are to the professional programming world. So I'll attempt to make some 2D games I think, and might have a go at making some sort of basic educational software so I can tie in all the physics I've learned. This will be coupled with continued study into the areas that I think need more brushing up, e.g. OOP etc.

Also I'm working through everything on Code Academy, just to get a taste of other areas. Currently learning Javascript.

Oh and I'm thinking, after my bachelors I'm going to try shifting over to a Computer Science or Software Programming masters of some kind, if possible. 

Aside from that, any pointers for how to actually get into this as a line of work would be much appreciated.

Thanks.


----------



## HAL (May 10, 2014)

Also I'll add - if my chances appear minimal.... no worries, you can be honest and tell me straight.

I'll just become another shill and join the finance world. The other industry that tends to hoover up most physics grads... heh.


----------



## ninjahitsawall (Feb 1, 2013)

I dunno how helpful this is because I wasn't a physics major, but maybe a certification of some kind? I'm looking more into data science (also doing a data mining project as part of an internship), so I am taking a SQL course, considering Python in the future b/c I'm learning that too now, and getting certified in one or the other (maybe both, in a perfect world, but I have undergrad debt to factor in too, haha). They're often scheduled in a way you can also be working at the same time, and/or they assist you with finding work.

I've noticed technical jobs do value certifications more highly than employers, generally, regard a Bachelor's.


----------



## dizzycactus (Sep 9, 2012)

There's no real need to make a personal portfolio so long as you've got some kind of qualification that shows you can code. I mean, I guess it wouldn't hurt, but I've always been a somewhat lazy sort. 


Feel free to ask about any programming topics you're not clear on. Most people aren't good at explaining things, because they tend to explain it terms of what something does and how it acts, instead of what it is. Take functions, for example. It's basically just a pre-written block of code you can use over and over again. But you can also pass it information to customise how the code will run exactly. So you could have a function that is "open door", but you can tell it which door to open. So the behaviour is fixed, but the data isn't. Or on a less abstract level, you do the same thing with the same variables, but you can tell the function which value the variables will have for a given call. 

I'm guessing you know what functions are, though. Well, polymorphism is just when you have more than one function with the same name, then you can tell what function you're calling so long as the variable types are different. If you have a function that is:
void doStuff(int x);
and:
void doStuff(double x);

And you call it like:
double x = 9.1;
doStuff(x);

Then it'll know to call the function that accepts a double, even though it has the same name as the other one. So, it's actually really simple.


----------



## nO_d3N1AL (Apr 25, 2014)

My advice to be to do some interesting projects, I found that helped me out a lot. Try to develop some utility or application which is not only commercially useful (in theory) or at least a time-saving piece of software, but also demonstrates your understanding of a range of concepts and issues in software development and computing in general.


----------



## Caveman Dreams (Nov 3, 2015)

This company I used to work for used to employ Physic's Grad's to do their betting engine, it was top of the art (no bragging, being deadly serious, company made billions, not millions), they also employed maths graduates to do this work as well. I would never of thought of the Gambling Industry using this sort of technology. But it calculated bets, odds, changes in odds, etc.

They were more concerned about emplyee's maths skills than programming skills.

I would recommend googling Harvard CS50, and doing their free course (https://www.edx.org/course/introduction-computer-science-harvardx-cs50x).

This covers a lot of programming, even programming your own mail server. So I believe it would be a foot in the right direction.

If your in the UK, PM me and I will give you a link to the company I used to work for and see if there are any jobs of that type going.

It seems more interesting than a typical software house, that releases software then developers spend their life just creating patches. (Worked at 2 such firms, 1 was good, 1 sucked). 

Anyway good luck. 

Also try GitHub and start writing code and fixing other peoples code, create a portfolio.


----------



## Parade of Sparrows (Mar 16, 2010)

Proving your ability through your knowledge and experience seems to be what's important in the industry in my area (mid US). I did not graduate from college but somehow worked my way up through software tech support to manual QA, then automated QAE, to front end developer, and I'm working on server side automated QAE at the same time. 

From my experience entry level programming job candidates in my are seem to have very little actual knowledge that we are actually looking for even with a degree. Although having the degree from school is definitely a leg up. 

Below is what I think are the most key pieces of information to know that seem to be in high demand. (this is a very personal experience based list)

Object oriented programming languages with an understanding of concurrency
SQL SQL SQL SQL SQL 
Javascript
bash scripting/command prompt/powershell scripting
Agile vs Waterfall development methodologies. 



Bonus points if you understand the difference between monolithic applications and microservices based distributed systems. 
Here's a clue Microservices ftw or for the money.

If you can leverage your knowlege from your programming classes and answer their specific language based questions for entry level jobs you can land the job in multiple companies in my area. 

The company I am at right now I have trained multiple kids out of college that could not do fuck all when it comes to actual real world programming work.


----------



## dizzycactus (Sep 9, 2012)

talon235 said:


> Proving your ability through your knowledge and experience seems to be what's important in the industry in my area (mid US). I did not graduate from college but somehow worked my way up through software tech support to manual QA, then automated QAE, to front end developer, and I'm working on server side automated QAE at the same time.
> 
> From my experience entry level programming job candidates in my are seem to have very little actual knowledge that we are actually looking for even with a degree. Although having the degree from school is definitely a leg up.
> 
> ...


My experience with academics vs experience wasn't so much that we lacked language of the knowledge itself, but that we lacked knowledge of the surrounding infrastructure. 

One glaring example - our course taught us java as the main language, but all we did was put java into an IDE and press the magic "run" button. So we didn't even really understand what a language is, and how it relates to a program. We never even encountered the idea of a compiler, or makefiles etc. Of course, being Java, this wouldn't be completely applicable anyway, but still. There was a fair learning curve entering my job three years ago when I hadn't even used a Linux shell before. Only took a few weeks to become comfortable with everything new, though, so I think employers to some extent emphasise previous knowledge a bit _too _much. It doesn't take long to learn something new if you've got a similar framework laid down from something previous.


----------



## Parade of Sparrows (Mar 16, 2010)

dizzycactus said:


> *My experience with academics vs experience wasn't so much that we lacked language of the knowledge itself, but that we lacked knowledge of the surrounding infrastructure. *
> 
> One glaring example - our course taught us java as the main language, but all we did was put java into an IDE and press the magic "run" button. So we didn't even really understand what a language is, and how it relates to a program. We never even encountered the idea of a compiler, or makefiles etc. Of course, being Java, this wouldn't be completely applicable anyway, but still. There was a fair learning curve entering my job three years ago when I hadn't even used a Linux shell before. Only took a few weeks to become comfortable with everything new, though, so I think employers to some extent emphasise previous knowledge a bit _too _much. It doesn't take long to learn something new if you've got a similar framework laid down from something previous.


The bolded portion is something that I want to emphasize. I agree wholeheartedly that whats lacking from newer recruits isn't knowledge of language but rather how those languages are used in the development of enterprise software and that isn't something that is taught in academia (I may be mistaken as I've only had limited exposure to these courses). I think too many people focus on not knowing enough about programming fundamentals when there is a real need in the industry for people who can learn things and apply what they learn in support of the business. 

I had a friend who graduated with an Electrical engineering degree who wanted to get into development that passed up multiple jobs that I was throwing at him out of fear of not knowing enough Java, or SQL to land the job. Multiple people who know way less than him have filled the positions. 

If they are a good company they will teach you how to use the knowledge you have acquired. What companies really look for with degrees is the ability to learn, apply, and adapt. So I would just say that if the OP is worried about his degree not applying to the job and thinks he needs to pick up another degree to get into the job he wants then its not a _*completely *_valid concern. 

Now if they just want to get a degree for their own interest in academics then more power to them. 

Our best front end developer got his degree in something related to art but he's able to utilize it as he helps in the design of the webpages we build. So a person can leverage themselves in many ways.


----------



## dizzycactus (Sep 9, 2012)

I just realised I said "lack language of the knowledge"
I'm not even drunk, I swear.


----------



## HAL (May 10, 2014)

NinjaBladesOfDoom said:


> I dunno how helpful this is because I wasn't a physics major, but maybe a certification of some kind? I'm looking more into data science (also doing a data mining project as part of an internship), so I am taking a SQL course, considering Python in the future b/c I'm learning that too now, and getting certified in one or the other (maybe both, in a perfect world, but I have undergrad debt to factor in too, haha). They're often scheduled in a way you can also be working at the same time, and/or they assist you with finding work.
> 
> I've noticed technical jobs do value certifications more highly than employers, generally, regard a Bachelor's.


Yeah I think some kind of non-degree certification would be good actually. I've also heard that Comp Sci grads are said to be strangely lacking in real-world programming skills (though can't say how true this is myself).

I did a specialist certification in something I'd probably still be stunted by the lack of real-world programming experience - hah - but having an exclusive certificate in something would always be a wee bonus and CV boost.



dizzycactus said:


> If you have a function that is:
> void doStuff(int x);
> and:
> void doStuff(double x);
> ...


Holy shit, that's insanely simple. I think I just stopped listening when we reached the point of polymorphism in my C++ course because we'd just thrown ourselves into objects and classes so I assumed anything after that would just be a whole other level of abstraction. So 'polymorphism' by name alone sounded like it would kill me. It wasn't a examinable aspect of the course so I filed it as 'not in important right now' in my brain.

Nice to know that it's much simpler than I thought. Thanks for the explanation.



nO_d3N1AL said:


> My advice to be to do some interesting projects, I found that helped me out a lot. Try to develop some utility or application which is not only commercially useful (in theory) or at least a time-saving piece of software, but also demonstrates your understanding of a range of concepts and issues in software development and computing in general.


I'm looking into this kind of thing now.

I was going to do Python stuff but I think C++ is better, so I've just downloaded Visual Studio and SFML so I can have a bash at making some simple 2D games.

Also gonna see about making some sort of basic utility program, or something loosely educational using the stuff I'm learning on my degree. Might even try leaning towards Java/Javascript (not sure which one is specifically required) for making a rudimentary phone app.



Reality Check said:


> This company I used to work for used to employ Physic's Grad's to do their betting engine, it was top of the art (no bragging, being deadly serious, company made billions, not millions), they also employed maths graduates to do this work as well. I would never of thought of the Gambling Industry using this sort of technology. But it calculated bets, odds, changes in odds, etc.
> 
> They were more concerned about emplyee's maths skills than programming skills.
> 
> ...


I had a look on GitHub but don't think I'm quite ready to dive into that level yet. Maybe I just haven't figured it out yet. Seems like it's all full of pros who know a shit-ton of stuff already.

About physics and maths grads doing gambling software - yeah it's the maths stuff we learn which gets us scooped up in that industry, I expect. Quantum mechanics and statistical physics (which deals with systems that have so many molecules you can only use probabilities to come to accurate solutions) gives us a pretty major grasp on the maths that would be involved in things like gambling.

The online Harvard course looks really interesting too, cheers.



talon235 said:


> Object oriented programming languages with an understanding of concurrency
> SQL SQL SQL SQL SQL
> Javascript
> bash scripting/command prompt/powershell scripting
> ...


Duly noted!


----------



## dizzycactus (Sep 9, 2012)

HAL said:


> Yeah I think some kind of non-degree certification would be good actually. I've also heard that Comp Sci grads are said to be strangely lacking in real-world programming skills (though can't say how true this is myself).
> 
> I did a specialist certification in something I'd probably still be stunted by the lack of real-world programming experience - hah - but having an exclusive certificate in something would always be a wee bonus and CV boost.


Hmm, both of our universities have an MSc in software development. We're both from the UK (I think?), and you use "wee", usually narrowing it down to Scottish or Northern Irish. Is it possible you're actually at the same university I was at, planning on doing the same course? lol 
I went to Queen's, Belfast.


----------



## HAL (May 10, 2014)

dizzycactus said:


> Hmm, both of our universities have an MSc in software development. We're both from the UK (I think?), and you use "wee", usually narrowing it down to Scottish or Northern Irish. Is it possible you're actually at the same university I was at, planning on doing the same course? lol
> I went to Queen's, Belfast.


Good effort but nil points I'm afraid, heh.

I'm from northern England (North Yorkshire) and probably use wee because half my family are Scottish (not really been very much immersed in Scottishness overall though, so I must have just absorbed the word at some point in childhood).

At Leeds uni now.


----------



## Caveman Dreams (Nov 3, 2015)

HAL said:


> Good effort but nil points I'm afraid, heh.
> 
> I'm from northern England (North Yorkshire) and probably use wee because half my family are Scottish (not really been very much immersed in Scottishness overall though, so I must have just absorbed the word at some point in childhood).
> 
> At Leeds uni now.


Haha Leeds Uni, back to that PM, the company I mentioned do a Graduate Scheme (3 months at £21K, rising to £27K after 6 months or so) and are based in Leeds, that said its full of Corperate Political BS. I'm more of a Techie Guy so left the place and now doing Server and Network Cloud Engineering which suits me better. Cant stand Large Firm BS.

In terms of UK, London is full of opputunity.
Leeds is quite good.
Sheffield is more Development work (good for you)
Nottingham is mostly support work
Milton Keynes is basically Silicon Valley (theres a firm that builds Super Computers)

Plus you have BAE in Leeds as well who probably do Graduate Schemes.


----------



## dizzycactus (Sep 9, 2012)

Reality Check said:


> Haha Leeds Uni, back to that PM, the company I mentioned do a Graduate Scheme (3 months at £21K, rising to £27K after 6 months or so) and are based in Leeds, that said its full of Corperate Political BS. I'm more of a Techie Guy so left the place and now doing Server and Network Cloud Engineering which suits me better. Cant stand Large Firm BS.
> 
> In terms of UK, London is full of opputunity.
> Leeds is quite good.
> ...


I went to London once, on a work trip. I was worried I'd choke to death on the air pollution before I'd make it home.

Never liked cities though. Claustrophobic, hard to go anywhere or get anything done. Crowds, queues, no parking. At least London has the underground, which is fairly convenient.


----------



## Caveman Dreams (Nov 3, 2015)

dizzycactus said:


> I went to London once, on a work trip. I was worried I'd choke to death on the air pollution before I'd make it home.
> 
> Never liked cities though. Claustrophobic, hard to go anywhere or get anything done. Crowds, queues, no parking. At least London has the underground, which is fairly convenient.


Ive only been to London on nights out and visiting Houses of Parliament.

I do like Cities, but not sure I would cope with London, too expensive, even job offers I have jad there, although they would pay well further up north, in London its peanuts.


----------



## SmallData (Oct 29, 2016)

Reality Check said:


> I would never of thought of the Gambling Industry using this sort of technology.


Finance is one of the biggest (if not THE biggest) employers of math graduates. Most trading these days is done by machines and algorithms, not people. Ironically even these incredibly high paying jobs are being automated (in fact they are very EASILY automated). It's all a total sham though of course, these algorithms are just trying to game one another and waste Billions of dollars and many smart minds competing against each other while providing nothing of value to society. More countries need to pass financial transaction taxes to stop this practice.


----------



## dizzycactus (Sep 9, 2012)

SmallData said:


> Finance is one of the biggest (if not THE biggest) employers of math graduates. Most trading these days is done by machines and algorithms, not people. Ironically even these incredibly high paying jobs are being automated (in fact they are very EASILY automated). It's all a total sham though of course, these algorithms are just trying to game one another and waste Billions of dollars and many smart minds competing against each other while providing nothing of value to society. More countries need to pass financial transaction taxes to stop this practice.


 I work for a company that provides the intermediate software to trading firms like banks so they can get e.g. market data from exchanges, but I always wondered how the stock market was really contributing to society when it all seems like a zero sum game - if you gain money, it's because someone else lost it. I guess companies get money from issuing their shares in the first place, though. And the concept of dividends seems pretty sound.


----------



## SmallData (Oct 29, 2016)

dizzycactus said:


> I work for a company that provides the intermediate software to trading firms like banks so they can get e.g. market data from exchanges, but I always wondered how the stock market was really contributing to society when it all seems like a zero sum game - if you gain money, it's because someone else lost it. I guess companies get money from issuing their shares in the first place, though. And the concept of dividends seems pretty sound.


Well, you have to remember that the stock market of 2016 is a far cry for the original stock markets. I honestly could go on forever about the subject but given as I have limited time I'll just copy part of an email I send to someone else earlier today:

"Anyways, I continued my reading of books most cited by the Governors of Federal Reserve Bank and most recently completed "The Price of Inequality" by Nobel Prise winner Joseph Stiglitz. I'm currently simultaneously reading a book on trading options and "Weapons of Math Destruction" by Cathy O'Neil which deals with the way "Big Data" algorithms are used (or more specifically misused) in such a way that people's live are negatively affected due to correlations that are unfairly biased (an example being algorithms which determine whether someone should get parole and base such decisions on demographic parameters such as recidivism rates from the ZIP code the individual lives in and not on the specific individual in question). It's frustrating reading books on Economics and Data Analysis because it reveals just how blatantly broken the markets are. The existence of the market is supposed to be predicated on the idea that a free market is efficient in determining price (which in the case of equities should mean the best companies get the most money thereby allowing them to grow). However when you read a 300 page book on options and not once is it mentioned how to evaluate a company's value but instead every chapter is based on how to evaluate trends in the data it become quite clear that Keynes' was right when he called the markets a beauty contest in which the purpose in not to pick who the individual finds the most beautiful, but who the AVERAGE individual finds the most beautiful. This is not what markets are supposed to be about and the fact profits and losses are based primarily on predicting the psychology of the market and not the underlying data means the market is in fact profoundly inefficient at assigning price and risk. It's more than just the market inefficiency that is problematic though; it's the fact that money can be made much more readily in the finance industry than it can be in most any other industry. In theory the finance industry should be as small as possible since it performs only a support function and not a direct benefit to individual citizens, but instead it has a grossly outsized importance drawing both human and financial capital away from productive industries and into finance which in itself produces nothing and exists only by shaving away a few percent here and there from the actual productivity of the Economy. It's profoundly unfair that a blue collar worker who adds $200,000 of value a year to the Economy can be compensated at $30,000 a year whereas a futures trader who adds $0 a year can be compensated at $500,000 a year. What's perhaps more frustrating is the "Just World Theory" believed by such people whereby their success is entirely warranted due to their superior intellect. Being smart is great, but being smart at tricking people out of their money is entirely different than being smart at developing a new cure to cancer or a similar endeavor. It's in this regard that I think we need to worry about a Hillary Presidency because she has clearly been cognitively captured by the banking industry and I feel it is very unlikely that all her talk of increased regulation will lead to anything of substance in that regard. The regulatory framework which contributed to the "Great Recession" still exists almost entirely intact today and the historically low interest rates provide a ripe soil for the creation of further bubbles. Without effective regulation we're only going to head right back into economic turmoil because the markets favor short term risk taking over long term stability and it's only a matter of time before those risks come back to haunt us once again."

It's funny, just a few days ago a news article quoted the wrong stock ticker for a story that brought bad news about a company and automated algorithms parsing the news sites started selling off the wrong stock crashing it's value. It's stories like this that tell you just how divorced from reality the stock market has become. It's all a race to be a millisecond before the other guy and the result is massive instability as seen in several recent "flash crash" scenarios. High speed algorithmic trading should be outright banned or else taxed into nonexistence. Bernie Sanders was proposing such a tax but I don't think it made it into either of the current candidates platforms (of course it didn't, they are both from NYC which is built on these sort of market inefficiencies).


----------



## Jamaia (Dec 17, 2014)

@HAL Could you tell me why is it that we're learning Python? I'm on an introductory course to Python right now, it's a mandatory course for Physicists with a minor on methodology (or not sure how to translate the word, but meaning a collection of Chemistry, Maths, Programming courses, you know). And while I like the course very much, I don't know what it's for... Why Python and not some other language?


----------



## HAL (May 10, 2014)

Jamaia said:


> @HAL Could you tell me why is it that we're learning Python? I'm on an introductory course to Python right now, it's a mandatory course for Physicists with a minor on methodology (or not sure how to translate the word, but meaning a collection of Chemistry, Maths, Programming courses, you know). And while I like the course very much, I don't know what it's for... Why Python and not some other language?


I think basically it's a complex and broad-reaching coding language, with uses in many software applications, and is a _tiny_ bit easier than C++, so it's good for physicists because in science the main aim isn't really to make flashy games or whatever other things that C++ can do more easily, the aim is to make things that deal with and process/present data.

If you're interested, have a look at my Python project that I did last year. https://github.com/Stuart88/Physics_Muon_detector_coursework - click the 'download' button on the right and run the py15csa.py file. It might need to be run on a computer with Python installed on it though. I got a First for it so I guess I did something right.

The project basically takes a shit-ton of data results, like several thousand data points or more (so you can't exactly just hand type it into Excel), and processes the data. The most interesting thing is that the data is random, representing a physical process so if you plot the points there isn't a smooth curve, but you can use methods in Python to make it recognise patterns and output a mathematical function as the smooth curve that the random points represent. That's basically the endgame for a lot of physics experiments - seeing how the real world correlates to a mathematical understanding in some way.

Note the random blue points, and the absolute clusterfuck on the right hand side where the data gets extremely noisy. And note the green line which my coding managed to work out after analysing those points.


----------



## Jamaia (Dec 17, 2014)

Thanks @HAL, I looked at your project when you started this thread but I can't really follow the code, we've only used dictionaries and sets on our course so far . So I'm very impressed with your project especially if you feel like you don't even know that much about programming! We have some kind of a group project pending, but I don't know the details of it yet.


----------



## HAL (May 10, 2014)

Jamaia said:


> Thanks @HAL, I looked at your project when you started this thread but I can't really follow the code, we've only used dictionaries and sets on our course so far . So I'm very impressed with your project especially if you feel like you don't even know that much about programming! We have some kind of a group project pending, but I don't know the details of it yet.


Heh, I had to google quickly to see what sets are. I've never used them before. Looks like a form of data storage? That's basically all coding is really - storing/using/manipulating/outputting data.

I expect the only other important thing you'll learn is how to make a function. It's not too hard either. Functions are basically just a computer version of " f(x)= .... " 

Enjoy!


----------



## Jamaia (Dec 17, 2014)

HAL said:


> Heh, I had to google quickly to see what sets are. I've never used them before. Looks like a form of data storage? That's basically all coding is really - storing/using/manipulating/outputting data.
> 
> I expect the only other important thing you'll learn is how to make a function. It's not too hard either. Functions are basically just a computer version of " f(x)= .... "
> 
> Enjoy!


Ha, well functions I know, it's just that we're still at the very basic level with the coding, nothing has really been put together. Like one of the assignments just this week was to write a function that gets lists as an argument and returns a single list with all duplicates removed, and the solution was to convert the combined lists to set and back to a list (because sets only hold unique items). I'm more lost with all the operators and slow to figure out how to loop. But it's not like I'm the best student in the class, haha.

I was just getting a bit worried that maybe this Python thing is just something that Universities teach for some unpractical reason, because on our course there are a lot of referrals to other languages, like "in Java this is done a bit differently" and our latest substitute teacher even said he doesn't like to touch Python unless he absolutely has to, lol. I think our primary teacher does use Python in his work, though.


----------



## HAL (May 10, 2014)

Jamaia said:


> Ha, well functions I know, it's just that we're still at the very basic level with the coding, nothing has really been put together. Like one of the assignments just this week was to write a function that gets lists as an argument and returns a single list with all duplicates removed, and the solution was to convert the combined lists to set and back to a list (because sets only hold unique items). I'm more lost with all the operators and slow to figure out how to loop. But it's not like I'm the best student in the class, haha.
> 
> I was just getting a bit worried that maybe this Python thing is just something that Universities teach for some unpractical reason, because on our course there are a lot of referrals to other languages, like "in Java this is done a bit differently" and our latest substitute teacher even said he doesn't like to touch Python unless he absolutely has to, lol. I think our primary teacher does use Python in his work, though.


Ah right, well you're well on your way then!

To be honest I'm not a gigantic fan of Python either. Just look at most jobs out there - most of them mention some form of Java or C++, so it makes you question what Python is all about.

I think the bottom line is that Python is best for science stuff or, at least, it's best for scientists to get a quick grasp and make use of it for their requirements.


----------



## Tropes (Jul 7, 2016)

HAL said:


> Personally, I don't think my skills are anywhere near up to scratch. I've uploaded the Python project I did to GitHub so anyone with any knowhow can tell me where I'm at so far. Here it is: https://github.com/Stuart88/Physics_Muon_detector_coursework
> 
> I'm going to start some C++ and Python projects just to build up a bit of a portfolio but I don't know what standard these should be either. It would be embarrassing to create a load of pathetic, amateur projects without realising how simple they are to the professional programming world.


I am in a similar position, and haven't quite made the jump for personal reasons, but I'll give you an advice someone who has made it through gave me:

1. Yes, you suck.
2. In programming, so does everyone else.

We all go through the same process, we all jump back and forth between our projects the documentation and stackoverflow and it's kin, and when you learn more you do it even more often because the specific details and syntax will be more easily confused in your mind with other similar patterns, over time you'll start going back and forth to your own older projects trying to take pieces of what you've done there. There is a degree of awkwardness in playing with code and it's alright, what companies will look for when they see your code is knowing what you can do and getting a feel for your way of thinking and what they can gain with it.


----------



## dizzycactus (Sep 9, 2012)

Jamaia said:


> Ha, well functions I know, it's just that we're still at the very basic level with the coding, nothing has really been put together. Like one of the assignments just this week was to write a function that gets lists as an argument and returns a single list with all duplicates removed, and the solution was to convert the combined lists to set and back to a list (because sets only hold unique items). I'm more lost with all the operators and slow to figure out how to loop. But it's not like I'm the best student in the class, haha.
> 
> I was just getting a bit worried that maybe this Python thing is just something that Universities teach for some unpractical reason, because on our course there are a lot of referrals to other languages, like "in Java this is done a bit differently" and our latest substitute teacher even said he doesn't like to touch Python unless he absolutely has to, lol. I think our primary teacher does use Python in his work, though.


The tricky part is working out how to do it efficiently. You could just go through the list, and at every item, loop back over the items you've already gone through, and if you find a previous item the same as the current item, then remove the current item. But that's not too efficient. Off the top of my head, some sort of efficient sort function might help, because then duplicate items will be beside each other, so you only need to compare the current item with the item next to it, but it depends on how efficient the sort would be. It changes by length, so maybe the standard solution (below) for small datasets, a sort-remove hybrid for larger sets. Then which sort algorithm is most efficient itself varies by list length. 

The inefficient solution in a C-like language would be like:


```
for(int i = 0; i < list.size(); i++)
{
    string element = list[i];  //avoid re-retrieving list[i] each time in the y-based loop
    for(int y = 0; y < i; y++)
    {
         if(list[y] == element)
         {
              list[i].remove();   //dunno what the exact function for removing would be here but you get the idea
              break;               //a little more efficiency
         }
    }
}
```
You mentioned something about multiple lists, but not sure exactly what that entails. Above would be for a single list.


----------



## Jamaia (Dec 17, 2014)

dizzycactus said:


> The tricky part is working out how to do it efficiently. You could just go through the list, and at every item, loop back over the items you've already gone through, and if you find a previous item the same as the current item, then remove the current item. But that's not too efficient. Off the top of my head, some sort of efficient sort function might help, because then duplicate items will be beside each other, so you only need to compare the current item with the item next to it, but it depends on how efficient the sort would be. It changes by length, so maybe the standard solution (below) for small datasets, a sort-remove hybrid for larger sets. Then which sort algorithm is most efficient itself varies by list length.
> 
> The inefficient solution in a C-like language would be like:
> 
> ...


Here's what I did with Python: 

def uniqList(first, second):
print "Single list, no duplicates:"
return list(set(first + second)) #combines the lists with +, converts to set, converts to list

And calling the function, with two lists:
>>> uniqList(["a", "b", "c", "d", 1, 2, 4, 5], [1, 2, 3, 6, "a", "b", "w"])
returns:
>>> Single list, no duplicates:
['a', 1, 'c', 'b', 4, 'd', 6, 2, 3, 'w', 5]

Or calling it with strings:
>>> uniqList("1235fv mjiyr", "1346098jnmewq")
Single list, no duplicates:
[' ', '9', 'e', 'f', 'i', '6', 'j', 'm', 'n', '1', '0', '3', '2', '5', '4', 'w', 'v', 'y', '8', 'r', 'q']


----------



## dizzycactus (Sep 9, 2012)

Jamaia said:


> Here's what I did with Python:
> 
> def uniqList(first, second):
> print "Single list, no duplicates:"
> ...


Yeah because the set class implements a way to make the list unique, so it's basically telling something else to do your work for you instead of developing a solution yourself lol. 

Ideally you'd make your function accept a list of lists then loop through them to build the final list to be converted to the set. Then you can use any number of lists with it. Or what if the function just accepts a single list, but you call it like uniqList(list1+ list2 + list3) etc. 

I don't know much about python though, looks like a messy/loose hipster language. I like my curly brackets. 
About half my work is done in TCL too which is a quirky scripting language.


----------



## Jamaia (Dec 17, 2014)

dizzycactus said:


> Yeah because the set class implements a way to make the list unique, so it's basically telling something else to do your work for you instead of developing a solution yourself lol.


Yes, haha!



> Ideally you'd make your function accept a list of lists then loop through them to build the final list to be converted to the set. Then you can use any number of lists with it. Or what if the function just accepts a single list, but you call it like uniqList(list1+ list2 + list3) etc.


This was another way I thought of, without sets. Seems to work:

def uniqlist(list):
>> uniqlist = []
>> for i in list:
>>>> if i not in uniqlist:
>>>>>> uniqlist.append(i)
>> return uniqlist




> I don't know much about python though, looks like a messy/loose hipster language. I like my curly brackets.
> About half my work is done in TCL too which is a quirky scripting language.


hipster language XD!


----------



## dizzycactus (Sep 9, 2012)

For some reason I feel like posting about hash maps. Partly because they'd be a way to make my solution more efficient. Instead of the internal loop, we could just build a hash map of the list as we go along, and then check if the current item is in the hash map. No need to loop inside the main loop. And it's important to know about them. 

So, lets say you have an array/list, and you want to search through it for an item with a particular value, instead of getting an item at a particular place in the list. 
You could look at every item in the list and check its value until you find the item you're looking for. But that's really slow. 

Lets say we have a small library, with like 20 books. Each book starts with a different letter. We have 20 numbered spots on our shelf to put the books on. A way to store them that gives fast retrieval would be to make a kind of system that converts the book's name to the number of the shelf it's on, and then just go straight to the right shelf and get the book. So the obvious system is that the book that starts with 'A' is on shelf 1, 'B' shelf 2 etc. 

In programming terms, this is called a hash function. We take the value of an item, and convert it to a number which represents where we've stored it. Then when we need the item again, we put its value through the hash function to get its location, and go get it. 
This means we can look it up quickly in a long list - the main work is the hash function itself. Obviously computing a hash function is quicker than looking through like 10 million items for the right one, but looking through 5 items might be quicker than getting the hash function of an item in that list, so it's worth it for larger lists only. 

What happens if we store 2 books at, say, shelf 1? Then we return both books, and check both, and return the right one. If it gets to the point where each shelf has too many books, it's time to redesign the hash function to handle things better.


----------



## Jamaia (Dec 17, 2014)

I don't get it, @dizzycactus.


----------



## dizzycactus (Sep 9, 2012)

Jamaia said:


> I don't get it, @*dizzycactus*.


Any particular parts that aren't clear?


----------



## Jamaia (Dec 17, 2014)

dizzycactus said:


> Any particular parts that aren't clear?


I tried to form a question but I didn't get it enough to form a question .

Ok maybe start with is this still related to the removing duplicates problem?

And how about this here: 



> Lets say we have a small library, with like 20 books. Each book starts with a different letter. We have 20 numbered spots on our shelf to put the books on. A way to store them that gives fast retrieval would be to make a kind of system that converts the book's name to the number of the shelf it's on, and then just go straight to the right shelf and get the book. So the obvious system is that the book that starts with 'A' is on shelf 1, 'B' shelf 2 etc.


Why do you need hash-anything for this, if you know what you are looking for, then why not just have a list of the books and the index = shelf number? Like a list.

Oh is the problem that if you have 10 million items, even if you know the name of what you're looking for, the computer would start from the first one and check if it is a match and if not if the 2nd is and so on? Instead you'd need a system, something that would be like alphabetical order, where you can narrow your search down based on information you get directly from the title of the item you're looking for. Is that it? And is the "2 books 1 shelf" situation like looking for something from a list of lists?


----------



## dizzycactus (Sep 9, 2012)

Jamaia said:


> I tried to form a question but I didn't get it enough to form a question .
> 
> Ok maybe start with is this still related to the removing duplicates problem?
> 
> ...


Yep. If you're looking for something in a hash map (instead of a list) then it does it quickly because it just has to take the value, put it through the hash function, get the item's location, and go get it. 


> And is the "2 books 1 shelf" situation like looking for something from a list of lists?


We've established that checking every single book for the right one is inefficient, right? Well, if you arrange them alphabetically, then you still have to check every book in that section, but you have fewer books to check. A hashing function is similar in that several items can be returned with the same hash, but you have fewer items to search for. The hashmap itself does all this though, you don't have to search yourself. 

Here's an example of a hash map in C++:

```
unordered_map <string, int> m;
m["foo"] = 42;
cout << m["foo"] << endl;
```
There might be some notation there you're not familiar with. All you need to know is that each item in this map has a name, and a value. When looking for an item, we give the item name, and it returns the item's value. "cout" just prints to the console. So here we would print "42". If we re-write my solution to the previous problem with this, we have:


```
unordered_map <string, int> map;


for(int i = 1; i <= list.size(); i++)
{
    string element = list[i - 1];  //avoid re-retrieving list[i] each time in the y-based loop
    if(map[element] == 0)   //I believe if we can't find an item it'll just return 0
    {
        map[element] = i;  //insert into map
    }
    else
    {
        //duplicate, remove
        list.RemoveAt(i - 1);
        i--;   //need to move backwards one since we've deleted this list element
    }

}
```
But of course in this example it's a little awkward because if we can't find something in the list it returns 0, but because our list starts at index 0 it's possible we could have found something that was at index 0, so we had to change things a bit so we store e.g. element[0] with an integer of 1, element[5] with an int of 6 etc to accommodate. But you can ignore that and just look at the bigger picture.


----------



## Jamaia (Dec 17, 2014)

dizzycactus said:


> Yep. If you're looking for something in a hash map (instead of a list) then it does it quickly because it just has to take the value, put it through the hash function, get the item's location, and go get it.
> 
> 
> We've established that checking every single book for the right one is inefficient, right? Well, if you arrange them alphabetically, then you still have to check every book in that section, but you have fewer books to check. A hashing function is similar in that several items can be returned with the same hash, but you have fewer items to search for. The hashmap itself does all this though, you don't have to search yourself.


If the books are arranged alphabetically you wouldn't check every book. If you're looking for a book called ABCD, you'd check the first letter and go to the section A (or 1), check the second letter and find shelf B (or AB-) (or 2) in section A ( = maybe 1.2 or A2), then you'd check the third letter, find the C part of the shelf (C or 3), and finally out of all the ABC- books you'd get the one at place 4=D. Or maybe 4.0, to differentiate between ABCDA- etc. books. So if your book is ABCD, you get a code, something like A2C4.0 which is the location where you find the book. Or just simply ABCD (section A, shelf B, part C, slot D).That's not a hash because it's easy to convert back to ABCD, right? And it's not any faster to go through than the original title, because thera are just as many or more characters. 



> Here's an example of a hash map in C++:
> 
> ```
> unordered_map <string, int> m;
> ...


 I didn't know there's a [ CODE ] thing too  . Ok you're right I can't read this notation too well, but I don't see the function. Do you assign the value 42 to "foo" by hand? 


> ```
> unordered_map <string, int> map;
> 
> 
> ...


Haha, I can't read this either but I'm curious. Can you walk me through what happens here? I only understand there's a for-loop but I don't know what is in the parenthesis. Is it starting at i=1, loop until index is the same as length of the list, just meaning loop through the list? I don't know what is i++. Just add one to i with every rotation? Then within the loop you're giving a name "element" to what ever is in the list at index i-1, so, loop starting at list[0]. I don't know what is a y-based loop. Then there's if-else statement. If search for "element", or list[i-1] in the map returns zero, then add "element" to the map and give it value i, so 1. Or else, in case it is found in the map, remove it from the original list. Then loop back, i=2, element equals what is in the list at index 1... No you have i-1, so your loop continues at list[0], and checks it again.

But if I'm understanding it correctly I still don't get the point of the hashing . Why do you need a map that holds a value to the element? Because when you go through the map, in the for-loop, searching if map[element] is found, you go through all the things you have in the map anyway, no?



> But of course in this example it's a little awkward because if we can't find something in the list it returns 0, but because our list starts at index 0 it's possible we could have found something that was at index 0, so we had to change things a bit so we store e.g. element[0] with an integer of 1, element[5] with an int of 6 etc to accommodate. But you can ignore that and just look at the bigger picture.


Ok should've read this first but I guess I didn't entirely misunderstand the code. Just the bigger picture, lol.


----------



## dizzycactus (Sep 9, 2012)

Jamaia said:


> If the books are arranged alphabetically you wouldn't check every book. If you're looking for a book called ABCD, you'd check the first letter and go to the section A (or 1), check the second letter and find shelf B (or AB-) (or 2) in section A ( = maybe 1.2 or A2), then you'd check the third letter, find the C part of the shelf (C or 3), and finally out of all the ABC- books you'd get the one at place 4=D. Or maybe 4.0, to differentiate between ABCDA- etc. books. So if your book is ABCD, you get a code, something like A2C4.0 which is the location where you find the book. Or just simply ABCD (section A, shelf B, part C, slot D).That's not a hash because it's easy to convert back to ABCD, right? And it's not any faster to go through than the original title, because thera are just as many or more characters.


The exact tactics in implementing the hash function to be most efficient is not central to the purpose of a hash function and the basic concept; it only clouds things. 

In our library, we just arrange alphabetically. So we want a book beginning with 'A', we go to the 'A' shelf, there's 5 books there, so we look at each one to find the one we want. 
This isn't central to the idea of a hash function either, it was just an aside about how the hash function can't always resolve to a single result. It's not too important. 






> I didn't know there's a [ CODE ] thing too  . Ok you're right I can't read this notation too well, but I don't see the function. Do you assign the value 42 to "foo" by hand?


When you created a set, what you really did was instantiate an instance of a class i.e. an object. A class is a construct that holds data and has internal functions to act on the data and return results. We create a map object. We put data into the map object similar to how we put data into the list object. If you have a map with a hundred entries stored in it, and you ask it to retrieve the entry with a particular value, internally it calculates the hash of the value to find which "slot" internally the entry is stored in, then retrieves it for you. This is efficient for looking things up, but inefficient for e.g. cycling through the whole map. The idea of different data structures being efficient for different scenarios is one of the main things employers test knowledge of when hiring, so it's just worth it to be aware of a few principles as early as possible. 



> Haha, I can't read this either but I'm curious. Can you walk me through what happens here? I only understand there's a for-loop but I don't know what is in the parenthesis. Is it starting at i=1, loop until index is the same as length of the list, just meaning loop through the list?


Yes, but lists/vectors etc start at index 0 so usually we do from i=0 to i<list.size(). This is a special case because it would have broken my logic here to do that. 


> I don't know what is i++. Just add one to i with every rotation?


It's basically short hand for "i = i + 1", so yeah. You stop thinking about it after your thousandth loop or so lol. 


> Then within the loop you're giving a name "element" to what ever is in the list at index i-1, so, loop starting at list[0]. I don't know what is a y-based loop.


I was referring to my earlier solution which had a loop where I called the index "y". If you compare that solution to this solution you'll see how the previous one was inefficient. 


> Then there's if-else statement. If search for "element", or list[i-1] in the map returns zero, then add "element" to the map and give it value i, so 1. Or else, in case it is found in the map, remove it from the original list. Then loop back, i=2, element equals what is in the list at index 1... No you have i-1, so your loop continues at list[0], and checks it again.
> 
> But if I'm understanding it correctly I still don't get the point of the hashing . Why do you need a map that holds a value to the element? Because when you go through the map, in the for-loop, searching if map[element] is found, you go through all the things you have in the map anyway, no?


Nope, the map is a hash map, so it hashes the key value to get the stored value. It's efficient at single lookups. 
In my previous solution, instead of a map lookup, you'll see I had to loop back over the elements I had already traversed. You can imagine that if we have a list that's a thousand long then by the time I'm at element 900 I need to loop back over 899 elements, then at element 901 it's 900 elements I need to loop back over. As the list grows in size that we're checking for duplicates, the effort involved in removing the duplicates grows much faster than the size of the list does. The total internal loops done in a list of size 5 would be 1 + 2 + 3 + 4 = 10. For a list of size 10 would be 1 + 2 ... 9 = 45. We doubled the size of the list to check duplicates in, but more than quadrupled the work involved.

Now that I think of it, because we change the position of things in the list by removing from the list, I'm not sure my solution would work, because the hash map is assuming the indexes for each value remain constant... but that's programming for you. Make a solution, it fails, tweak it until it works lol.


----------



## Jamaia (Dec 17, 2014)

@dizzycactus I'm catching on but I guess I don't know how maps work. If in your map the value is I, how does the search map[element] find the value without going through keys until element is found and its value checked. Is it instead that search for element -> hash ->matching value? I'm at the boob lecture now, so may sound stupid. (More so than normally )


----------



## Jamaia (Dec 17, 2014)

Oh because map is not ordered, it's not searched one by one? ( dictionary in Python, I think)


----------



## dizzycactus (Sep 9, 2012)

Jamaia said:


> @*dizzycactus* I'm catching on but I guess I don't know how maps work. If in your map the value is I, how does the search map[element] find the value without going through keys until element is found and its value checked. Is it instead that search for element -> hash ->matching value? I'm at the boob lecture now, so may sound stupid. (More so than normally )


Boob lecture? Sounds like a strange course. 

This probably won't be completely accurate, but imagine the map has a list of 20 pre-allocated slots (we'll call it "myList"), and the hash function is just some sort of alphabetical thing.

So you say 

```
i = map.Get('D');  // (we can use [] but this makes it clearer in terms of that it's actually calling functions)
cout << i;             //prints whatever number is mapped to 'D'
```
for example, so internally it does something a bit like:

```
int Get(char x)     //a function that takes a char, returns an int
{
    int slot = hash(x);   //in this case call hash with 'D'
    return myList[slot];
}
```
So map "owns" "myList" and acts on it via a hash function that makes retrieval efficient.

Maybe when we're adding an element to a map it goes like:


```
void Add(char key, int value)
{
    int slot = hash(key);
    myList[slot] = value;
}
```
This isn't exactly how it's implemented, but as a kind of crude hash map, it might actually work to an extent.


----------

