ponedjeljak, 27. veljače 2012.

OS Memory management

After a while, I'm back! I'm crazy and I'm gonna swear a lot. Lets start politely not to scare you away. IDK who's reading this crap anyway.


Anyway, it goes like this


Memory - a capability of a human to remember stuff. There are many kinds of memory - long term, shot term? Perhaps many other, but I cannot remember correctly. Because I haven't been there for a while. Learning about human memory.


However, I've been learning about (*DRUMROLL*) computer memory. Yes, all the volatile and non-volatile, capacitive and magnetic and flash and flip-flops and whatever. YES, these are just memory types and methods of storing data.


A volatile memory is the one that gets erased whenever the power runs out. So what does the memory store? Since we're dealing with computers, it's bits. Bits, or actually two-state "numbers" or "elements" or whatever that can only have two states. Ok, that doesn't mean "ON" and "OFF", although one can understand with less effort if it visualizes it like being ON and OFF.


But a computer cannot understand ON and OFF. It understands voltage. So let's say 0.8 V is "OFF" and 5 V is "ON". (It can go either way, but the inverted version is called reverse-logic). NEXT - Why are we doing this?


Because the computer uses something called the Boolean Algebra. It's the mathematical formulation of truth values of statements, set theory and few other stuff using only two values - TRUE and FALSE. If 0.8 V is OFF, that 0.8 V is FALSE - and 5 V the truth. To easily create mathematical formulations and statements, finally, we use binary numbers. That means we only have two number - a 'zero' and 'one'. 0 and 1.


The story is: Computers use mathematical formulations of Boolean algebra encoded as voltage to calculate results of mathematical operations with binary numbers.


To make these calculations computers need to store binary numbers encoded as voltages, and put them through something called "logical gates", aligned and ordered specifically to give results of mathematical operations - for example, adding. So, how to store numbers? There is an entire field of study that handles that - but that's currently useless stuff to bore you about.   
WOW,  Is it really that simple?
But damn, I'm going too deep into developer's point of view, and this is a practical blog so I must quit doing this shit :). So anyway, let's talk about something that might interest you. How does having a lots of memory helps YOU in YOUR everyday work? Let's start with an answer.


YOU: "So, I have 500 GB of memory. PC, Y U NO WORK FAST?"


What? What are you saying? Let me explain:


First, let's make sure that both of us know the different kinds of memory. That means you must learn what I tell you, and I'll tell you what someone else told me :). If you make the same mistake after reading this you weren't paying attention so GET A GRIP. So,


Let me ask you a question. ME: "How much memory your computer has?"


You answering my question with the answer in the previous title is VERY WRONG. Since I'm quite smart, and believe you don't have a clue about computers, I understand what you are telling me, but this is not what I asked.


You told me about your hard drive space. Hard drives are a permanent storage class memory that every computer has. It could be a hard disk, a solid state drive or flash memory or whatever your computer has. Specific thing about this kind of memory is that is doesn't lose data when the power runs out. So when you install a program, download stuff from the Internet, copy pictures from your camera, it all stays in the same place you put it after you shut your device down. And, your camera has some permanent storage too. It's the memory card (a kind of flash memory, for those who are asking) pictures go to when they are taken. It would be very stupid if they go away when you push the OFF switch, right?


I asked you about the kind of memory computer currently uses to do work. Since permanent memory access is very slow (in context of today's computers), computer needs some active working space to load data from permanent memory, keep it here for quick access, to and to temporary store the results of data processing. Most stuff from memory is only kept for quick access (or is a copy of data from permanent storage for quick access) so when the computer isn't running, the contents of that memory don't need to be kept. So working memory is actually a scratchpad - you keep your active work in it, and throw it away when you're done with it. Let's compare - if you have a bigger desk, you can keep a lot of papers and documents for quick access, instead going for the file cabinet on the other side of the room to search for the particular thing you need. So you only have to go to the file cabinet once or twice and after you're done, walk once to the file cabinet to store done work. If you don't have a large desk, it's gonna get cluttered and productivity (in computer terms, processing performance) will drop. Most people don't care about performance, consider it bullshit, and me an idiot for particular optimization details I do with my stuff, so let me tell you this - performance is king. If you don't like it, get a hike.


So, let me correctly answer my question. I have 8 GB of memory on my MBP. The legacy name for that is RAM (Random Access Memory). It developed as a requirement because the applications were getting bigger, and CPU registers and caches aren't enough for fast and seamless performance. And now, the more memory you have, the computer's multitasking capabilities are higher and performance intensive applications are faster. What? What are registers and caches? Y U NO KISS?
Take a look, see what I mean?
What next? Let's see some software side of the story. Hardware is here, and it defines the speeds, throughputs, latencies, memory sizes, modules and other crazy shit. But how to harness all that POWAH and put it to good use? How will a computer manage all that hard grabbing memory shit? Smoke that shit?
Seriously crazy shit.


Could be scary shit


The culprits lie in very advanced algorithms and systems that manage, optimize, assign, protect, cache, manipulate, release, limit, transfer, copy, read, write, defragment and warn the user about the memory's current state. It's all contained in the main program. The mother of all running programs. The father, grand controller and king of your pc. All your bits are belong to him. 


The operating system. Windows. Unix. And other insignificant operating systems in miniscule amounts.


So, the OS actually manages memory through something called memory virtualization. It takes the physical memory (RAM) and pagefile or swap space (a very large file that's imitating memory storage) on the permanent storage and combines it together, abstracting the entire memory for things that need memory, and those things are called applications. An operating system takes a block available storage from the virtual memory and assigns it to a running application, but it has super-duper fucking advanced algorithm shit that assigns it to the best location it can currently get. So depends on the workload, memory capacity and other elusive factors, it stores it to either physical memory or page file. Completely transparent to the applications. If you ask me, that's fucking cool.


Generally performance goes like this: If you have more physical memory, you can load more applications and multitask efficiently. But computers work with limited values, and memory slots are also limited. So what's the cheapest way of getting my PC work fast? If it's a PC, buy and install Windows 7 with 4 GB of memory. If it's a Mac, buy Windows 7 and Bootcamp your ass to it. Windows 7 is the swiss knife of computers - it has the best applications, it has fucking cool memory management systems and it's too fucking simple to use that it's not even a challenge, it's boring me to sleep. Aim for the 64bit version. 3 reasons:


- You can have more memory (more than 3.2GB effective)
- Applications don't have memory limitations that are feasible in the next 100 years.
- Tis generally faster in 99.9% situations. Unless it was written by a developer fag.


If you have a 32 bit OS, you are limited, both mentally and practically. Y U NO explain why? OK, hold on.


Every block has 8 bits (0 or 1, remember?) and it has it's own address so the computer knows where you stored data. The computer, if it works with 32 bit numbers and have 32 bit wide registers - or to generalize, can process 32 bit numbers, it can know about 232 memory addresses. And that's 4,294,967,296 blocks. If everyone of this blocks is 8 bit wide - or 1 byte wide - its 4,294,967,296 bytes of memory. And that's 4 GiB (notice the 'i' - that means it's divided by 1024, and not 1000. It's pronounced gibibyte - it's a binary system instead of SI decimal system). So the effective limit is 4 GiB for 32 bit. If you have 8 GiB, and the picture above I said i have 8 GB of memory - it's possible. Mac OS X "Lion" is a 64 bit computer so it can address more than 4 GiB of memory. 64 bit OS makes your computer go faster and multitask better. GO FOR IT!


HOMEWORK: how much memory can a 64 bit computer address? It's a very long number to write :)


The end is near


This blog post is long shit. Now, when it's done, I guess you know a little more about how memory behaves and works to our advantage, with a practical advice to upgrade to 64 bit Operating Systems. Chances are that you have a 64 bit capable CPU, so I ignored that part in giving you the advice. If you have a 32 bit CPU, than you don't have to concern yourself with this. My advice is to buy a new computer. All CPUs that you can buy as a general consumer today are 64 bit and even have multiple cores! Pair that with 4 GiB+ memory and your computer will go crazy bat shit fast. And pair that with an SSD and it'll go inhumanly insane shit fast. So the end of 32 bit computers and 32 bit operating systems are near. Windows 8 Server is cutting 32 bit support, and all better retailers are bundling 64 bit OS installed (you paid for the licence, don't worry. Windows 7 has the same licence number for 32 and 64 bit version, no sweat)


Oh and BTW, the end has come for a user to know squat about his/her computer. Now when computers are becoming more complex, and Operating Systems more specific, you need to know something. Otherwise you'll make stupid mistakes and have less then you paid for a computer that you paid much. Fight with knowledge. Good knowledge can make a cheap computer fight the big ones that have assholes using them.


Fear not the stuff inside that cool aluminium shell or a boring matte plastic box.

četvrtak, 20. listopada 2011.

About developer's lifestyle

I'm now just a year and two months in my professional software development career. But before that I was a PC superuser, so basically I know my way around computers... better then most computer users.

For me buying/setuping/installing/assembling/whatever with a computer is a simple and effortless task. Even software development, which is the hardest thing that I've been doing on a computer (next thing is recovering a Bob's crashed system ... full of malware), isn't really that hard because it's interesting and complex enough for it to be challenging but not impossible. And I like a challenge :)

EDIT: Basically fixing a crashed system IS the hardest thing. Developing is easy :)

In other domains outside of project management, software engineering and all computer related stuff however, my situation is not as shiny :)

One of those lacking things is the amount of real-estate I have. Nothing. I only have a car (a good one) :O

Buying one? I really think something is wrong there. I think this world is set up for it to be almost impossible if you are a normal employee of a normal company. Yes, I'm still early in my career (very early, but light years in the software engineering field, I'm afraid) compared to other jobs currently on the market, and the country I'm living in doesn't really respect my kind of work so my options are not that good. Actually, they underestimate it and call software engineers "weirdos" and "nerds". And that we only sit and type and click and make money. Because we're not selling something or lifting heavy objects doesn't mean a software engineer's jobs is easy.

A common misconception (in Croatia).

It's hard.

And it's damn hard if you do not understand exactly what you're doing and have excellentmental abstraction and concentration skills. Otherwise you'll just see a bunch of text connected with dots and a lot of brackets. We are not underpaid, but the salary doesn'tscale well. For example a software engineer in Croatia has double the salary of a shoe salesman or saleswoman, but works at least ten times harder. The mind quickly gets tired.

This is true with all creative and mental jobs. For example, cooks need to think hard about the ratios of the spices, ingredients and amount of food they need to cook. Hairdressers need to think how to cut the hair for it to look good, create a mental image, imagine the process how to get there, and not to make mistakes because you cannot undo over cut hair, or to have a wrong idea of the hairstyle the customer wants... hard stuff. And not valued right.

I'd give the cook at least ten times bigger salary then the shoe salesperson. I'd give the hairdresser at least ten times bigger salary then the shoe salesperson. I'd give anyone that needs to professionally concentrate more that four hours per day at least ten times bigger salary then the shoe salesperson.

But this is not the way the world works and I surely live in a dream world. So what I want most is to get the work valued right. The effort and the results. In the meantime - fail.

There's hope, still

In a hopeless underpaid world you cannot really buy your own real estate and enjoy it. You need to either have a respectable and profitable company that makes a lot of money, and after paying all the taxes, have enough money to buy yourself real estate. You need to work your ass off!!!!!
Or you inherit some. I'm not that lucky either. So the only option for me to have my own place is to work my ass off. That's not a problem while I'm doing the job I like. I can always rent a place and live there :). Buying one is not an option unless the two before-mentioned things happen. Never an option.

Having a rented place is very flexible, and I think it works great with my current line of work. A developer is mobileflexible and remote. As long as you type code, it really doesn't matter where you type it from. So you can be anywhere where there's an Internet connection. So I don't need a fixed place to live. Only a place to live. With an Internet connection, of course.

Decisions, decisions

Karma has it hard. Especially when I'm surrounded with people that do not understand our line of work. And that it's the kind of work that's not fixed in any way, but extremely agile. We don't need a fucking real estate to call our own. We only need a car to travel, and a place with the Internet connection. And that's OK :)

We will not be bound. That will be all.

Having more real-estate isn't always buying more

Something unprecedented happened yesterday - I somehow started liking my Hackintosh.

So how did this happen?

It's the virtual desktops. And the way you go around with them. Snow Leopard and Lion (as well their *nix Linux bastard relatives) have an option to use virtual desktops. That's probably one of the must underrated features that an OS Shell possesses. AND WINDOWS DOESN'T HAVE!

So me being a software engineer with woman-like power to coordinate multiple activities (proven), I can quickly "switch my context" between multiple activities and remember "my last state". Like app switching on mobile platforms :)

Virtual Desktops

It's like behaving that your place has more room than you actually see, and you are able to access it quickly.

So let's imagine a user called Bob. Bob is a standard PC user - there's a guy I know that is behaving like "Bob", so I'm going to use his behavioural pattern as a starting point. And Bob likes his computer. Because it can help him listen to music, chat using his favourite chat service, surf the web, check the mail, use Facebook, etc. So to do all those things Bob has to:

  • Have Chrome or a-browser-of-your-choice opened, probably with multiple tabs
  • Have a music player window open
  • Have a chat window (or windows) open
  • Have a mail application open
  • I dunno, maybe a word processor window open. Something.

So he already has 5 or more windows open. His idea was to do all this stuff in parallel. But even if he has a large screen (like a 27" screen, but he's Bob, so he probably doesn't have) the OS Shell main screen is going to get cluttered. And switching between apps will either make you move your mouse around a lot to switch apps in the app controller (Windows users call this "Taskbar"), or work your fingers with Alt + Tab or Command + Tab or whatever equivalent your OS Shell uses, but you may not get the app that you want immediately. And the time you need to access the application screen is around 1-3-5-7 seconds. It will give you the effect that you're racing against time if you quicky try to respond a chat message, update your status or change the song or read the news or ... well, whatever. And that's NOT OK, it can make you nervous.

Solutions? There are a few I know of.

  • Buy and connect another monitor. You'll have more space, and you can also use one monitor for frequently used stuff that you need to see all the time. Like Chat boxes, Facebook or Twitter updates, a movie (if you can work and watch a movie at the same time) et al.
  • Use OS Shell features to round up app windows. For example, in Windows 7 you can shake a window to minimize all other windows and the same action to revert. Cleans the fucking place :). Also see OS X Exposé, works hell of a lot better than anything Windows OS' offer.
  • And the crown jewel - virtual desktops.
I have a picture for you people:


This is the Mac OS X "Mission Control" feature in Lion. What does it do? It rounds up windows on all desktops. It's a combination of Exposé and Spaces. "Spaces" is the Virtual Desktop feature of the Mac OS X from the previous versions. So you round up windows on virtual desktops (by some rule, I don't know) and switch desktops by hitting keys on your keyboard or set a shortcut on your mouse. You can now see everything - and seeing everything gives you a great sense of omnipresence (a "god" feeling?). Powerful stuff. 

Before Mission Control you could combine Exposé and Spaces. That also worked great - but now you can save even more time since with the same keystroke you activate both window switching and desktop switching. And that's OK.

So basically, you create virtual desktops, put windows on them and switch them using shortcuts. It's different than switching application windows, because if you can align your windows on desktops (one time operation) you can switch multiple applications simultaneously using a single keystroke or operation. Now that's VERY OK :). I'm addicted to that. And Lion has it easiest. That's why I started liking *intoshes :)

GC.ReRegisterForFinalize(this);

And at the end, all I'm saying is that Bob can make his job easier if he decides to use virtual desktops. If he's a Windows user, I'm afraid he's stuck to mediocre virtual desktop apps that actually complicate things. But does that mean him buying a Mac? Or building a Hackintosh? Installing Linux? I personally don't give a falling-star's-ass about what he's going to do. I have 4 operating systems on my PC, 2 of them have virtual desktops, Bob is responsible for his actions - I'm just giving him a choice :)

Learn to switch context fast. The brain can work multiple stuff if you make it do it. It'll help you multi-task!!!

That would be all, Number One.

srijeda, 19. listopada 2011.

Hudsoning (The Start)

Today it's about Hudson.

Hudson is something called "the continuous integration server". It's used to automatically builds code projects (web applications, mobile applications, whatever) for what-ever-purpose-you-need. So you configure Hudson to build you a project from source code (or source repository).

What is it exactly?

It's basically an WAR file (a Java servlet executable?) you start on any PC that has JRE installed. It attaches it's web interface to a socket on your computer, at the port 8080. You can access it from anywhere using Chrome or a-browser-of-your-choice (avoid IE, it has some font rendering issues) if you know the hostname, so using:

localhost:8080

you connect to the it on your local machine. For example, using

somehostname.somedomain.com:8080 

you get to connect to it from anywhere where there's an Internet connection. If the WAR file or a service is running on that computer, of course. I'll go to the differences later. If you think that's OK :)

A little bit of history with me and Hudson

Running into this tool in my software engineer career was a coincidence. Since I started working for Where Inc. and it's Windows Phone 7 project, some Chinese testers configured a project on Hudson (very badly) to make builds of the app every 5 minutes if there's a change in the SVN repository. It worked good while we used the old repository and the branch naming scheme that was hardcoded in the Ant scripts (which I didn't have access to, so I discovered this when I finally got access). Also using Ant for building a .NET project? You must be joking. 

Next up, I was in charge of the project's technical details (the former software engineer in charge got transferred to a more important project). So as first order of business, I needed to fix the build server, because I changed SVN repositories and project structure. Of course, the build failed to build. And that's NOT OK.

I had a day to get the data of the server, learn Ant scripting, do a lot of RTFM-ing just to figure out what these Chinese folk were doing. Now imagine a turning rainbow wheel....


... after which I got it ready, finally. The general idea was to only hardcode the procedure. Source and destination directories, SVN revision numbers, build paths, changelogs and all other stuff was either given as an external enviroment variable through Hudson or using existing enviroment variables from Hudson. Like SVN version build. Or workspace path. And finally, I was ready to say "And that's OK". But not yet, it seemed to complicate itself.

The Chinese folk also made a small Java console application that modifies the AssemblyInfo.cs file and WMAppManifest.xml file that replaced the minor version and revision numbers with the SVN revision number. AND THAT APP WAS SO VERY MUCH HARDCODED. That most certainly wasn't OK.

Rewrote that small Java console app to use Java properties info file paths, AssemblyInfo.cs and WMAppManifest.xml file pats as command line arguments in precise order. And through Hudson I gave the Ant scripts those file paths which then were used by the Ant script to run the console app and modify those two files. Except the procedure, nothing else wasn't hardcoded. And finally, that was OK.

Branching? Easy. I cloned the Hudson project, changed the SVN repository path to point at the branch, and I got the build.

Migration to Git (TBC)

Interesting stuff happened 6 months after. We migrated to Git, and after transfering the project from SVN to Git, I had to transfer Hudson too. No problem... Really. Except for these two:

  • How to get the project from Github? It used the authentication method with a passphrase, and Hudson didn't have the option to input a user name and password, even after installing the plugin. FAIL.
  • How to get the revision numbers? Git uses hex hash numbers for commit numbers, while SVN uses a simple decimal number. FAIL^2.
I'm afraid I've already taken a lot of time from my working hours to write this so I'm going to stop for a moment. To be continued... to keep you reading more :)

GC.Collect();

Stay tuned for Part Two! I'll put screenshots of the Hudson setup with instructions to make a build of an WP7 Application. WOW, new stuff! 

Is it fun? It is for those who want to know. And that's OK :)

utorak, 18. listopada 2011.

Hackintosh and input lag

It's enormous!

You really cannot believe it's there. Even if I use a massive CRT monitor the mouse still lags (although not as much). I can really "feel" input lag because I was a FPS player and I can figure out the slightest twitch... pretty epic, but useless :)

But what's really strange, nothing in Windows and Linux.

Somehow I couldn't get to the root of the problem. USB drivers perhaps (kexts)? I tried installing Logitech drivers for the mouse, and the lag is less obvious but still there.

Then, VOILA! The problem is in the crappy mouse acceleration and pointer smoothing algorithm that does the job so badly you cannot really get the "feel" for the mouse cursor. The pointer behaves strange, compared to the Windows' or Linux's mouse smoothing method. After a simple shell command:

defaults write .GlobalPreferences com.apple.mouse.scaling -1

The acceleration is gone. Let's rejoice :))

Perhaps it's a matter of getting used to? Sure, a smart man wrote something about Windows vs. Mac Font smoothing system. Mac users like Mac smoothing, and Windows like Windows smoothing. Then that must be the case.

OFFTOPIC HERE:
I personally like Windows method. It's much sharper than the Mac OS X, and it's more readable on screen. There are many blog posts about the Mac OS X vs Windows, so I'm not really going into anything here. But of course, you must always see both sides of the medal. So I blame nobody, OK? And that's OK :)

BACKTOPIC HERE:
But I'm not really ready to accept that solution just yet. I'll continue research.

If that happens on a real Mac with Mac OS X and it's Magic Mouse (the lag and crappy acceleration method), then I'll personally call an Apple Genius guy or girl and explain him or her the Smoothing algorithm and the math behind all that to make him or her feel bad and buy me a real Mac!!!!1111oneleven

Maybe I just want a Mac? I won't say that I don't want one, but I just currently can't afford one. And that's NOT OK :)

On PC multimedia

So lucky me, I bought a soundcard (not really lucky, I spent $100 on it)

It's the one CNET said it's like, the best consumer sound card that you can afford (let's not include those that cost like, a $1000). I wasn't wrong when I bought it, but I'm not very satisfied with it.

Sure, It's a good card. The sound is great, the drivers are good. It's PCIe so I don't have to worry about future upgrades (thank you Intel for making this happen... albeit too late). Has ASIO out of the box, and the driver download was a few clicks in Chrome or a browser-of-your-choice.

So what's wrong, huh? Nothing really.

It's a Certain-Taiwanese-Company's Certain-Implementation of the CMedia Oxygen HD Sound chip. Sounds familiar ^^. It's that card...

BUT WAIT :)

Cons. Or what I call "the fun part" :). I'll be formal and objective here, so let's avoid Fanboyism.
  • Takes a floppy power supply. Not really a con here, but my cables got messed up in the case -.-
  • It doesn't work in Lion (yes, I have a Hackintosh, check my twitter. But more on that later)
  • It works in Linux. Works-ish, I had to waste my 5 Mbit Internet connection to get myself some PulseAudio control centers to fix Right Channel balance. Screw balance. I want to stream Youtube. It's multimedia. Damn.
  • The sound quality is not as good as it should be. It's better than the ALC888 on my board, but It's not THAT better. Perhaps it's the speakers... so no major placebo effect here.
  • I don't have $100 anymore. Could have bought me some decent shoes... Or go to Zagreb and back...by plane.
I'm not cheap, but really. Give me something better for the price.

And since I started exploring iOS development on my Crackintosh, I cannot play music or movies, or relax, take a break. No sound you see.

But me being smart (well, you have to see that in person), I had a Bluetooth USB dongle and connected the Bluetooth headset... the sound is awful, but at least it sounds. Also, the onboard chip doesn't really work good on a Crack-a-Mac, so basically I don't have ANY sound on speakers. And that's OK.

Boring mute Macrack. That actually annoys me most.

So after this rant, I played some music on my $100 soundcard using $150 speakers. Did I get motivation from the music? No. So that value's also gone. So what's with the extra power that the sound card needs to have?

blogPost.Finalize();


The moral of the story: If you want better sound on your PC, it's not just buying the soundcard. It's arranging your room that has the PC (so you can normally position the speakers), then buying good Hi-Fi speakers and THEN buying the soundcard to fit all those needs. The soundcard with which you can calibrate the output, set up the equalizer and do other Hi-Fi Voodoo and Mumbo-Jumbo.

Hi-Fi's overrated. For the most part.

I really just want some value for the price. And that's OK.

It's alive

So the blog is finally here. I picked Google blog because it showed up first @ Google. Oh and yes, I have an account.

If you continue reading, you might find some interesting IT stuff here... this is just the beginning.

Mostly, you'll find generic solutions to some standard issues found while developing apps and using the computer.

I'll try to keep it fun and educational :)

Thanks everyone! :)