ArthurGuy

Database Replication

25 February 2009 – 6:32 pm

For the past few months I have been working on a multisite IT project, part of it involves keeping various databases synchronized between different sites. A single master database needs to be pushed out to all the remote clients and a local database on each of the remote machines needs to be backed up to the master.
I hadn’t attempted anything as complicated as this before so it was a daunting task.

The only type of database replication I was aware of was the standard replication feature built into MySQL, I had tried to use this previously but had a lot of trouble as I found the different machines easily came out of sync and wouldn’t recover without manual intervention. Despite the previous problems I set up replication on the master machine and configured each of the remote locations to replicate a copy of the main database, this took a bit of work to setup but one it was working it seemed to operate reliably. When I had set it up I could randomly turn on and off machines and they would start replicating from where they left off, that was until I moved the master to a different location, then they decided to stop working.

Because I couldn’t have each of the remote sites copying there local databases to the master through replication I decided to use mysql dump on a batch job, I was originally going to create a script to dump the database and FTP it to the master and then have another script on the master which would import the backups as they arrived. Thankfully I discovered a brilliant little feature of mysql dump which saved the day.

Mysql dump normally outputs the dump to a file but you can pipe it to a mysql instance connected to another machine, this meant I could copy the database from the remote site to the master in one line. Once I had set this up it worked brilliantly; very reliable and very simple to use.

mysqldump --opt --user USER --password=PASSWORD DB | mysql --host=IP --user USER --password=PASSWORD -C DB 

After the continuing problems with the built in replication feature I decided to turn off replication and use mysql dump to replicate the databases out to the remote sites.
This obviously isn’t an ideal solution because as as the databases grow the amount of data needing to be transfered will increase, I believe it also locks the database as it dumps it meaning people can’t use the system as this is happening. Despite these problems it works brilliantly for my needs.

Incase your wondering the remote sites connect to the master site using VPN connections so I don’t need to worry about sending data over an un-secure network.

Posted in Internet | Comments

The iPod Touch

17 December 2008 – 12:01 pm

Last week I received an iPod touch, this is going to form part of my latest project so the University paid for it and I get to use it for a couple of months before the project goes live. I have never liked the iPod range and I was never the greatest fan of the iPhone, not because I thought they were bad devices but because Apple was and is so restrictive with everything, for example you can’t put your own videos or moves onto the devices.

When I received the iPod touch I was expecting to find it cool for a day or two before the novelty wore off but I was pleasantly surprised, the interface was far more intuitive that I expected and very responsive especially coming from a background in windows mobile devices. Connectivity was also my other concern, only having wireless an not mobile connectivity meant it was no more useful than my laptop as I could only get online at home or at work. Whilst this is a problem I find myself using the device a lot around the home, checking things online when before I might not have bothered simply because of the convenience, I even found myself watching a whole BBC program on iPlayer.

The device supports an Exchange connection so its brilliant for checking and replying to emails, the keyboard took a bit of getting use to but the auto correction software makes up for the inaccuracy of it, the only downside is the calendar software, even with the Microsoft Exchange link it leaves a lot to be desired.

I don’t think I would change my phone for an iPhone as I do prefer a proper qwerty keyboard but I can see myself having both devices. At the moment I am waiting for another Android phone to be released which will hopefully fix the flaws of the current one, namely no Exchange support and no support for bluetooth headphones, if this doesn’t come soon I may have to go for an iPhone.

Posted in Internet, Personal | Comments

KISS Media Player + Smoke = Problem

10 November 2008 – 12:49 pm

On Friday I was watching a DVD and all of a sudden the image froze and the player became unresponsive, I tried restarting the player but the system got stuck on the boot up process, it was at this point I noticed the odd smell and the smoke seeping from the DVD player draw. The DVD player is part of the Linksys KISS media player I bought last year, something I was hoping would last longer that 14 months.
With the player being just outside of warranty and not wanting to throw away a £180 piece of equipment I decided to open it up and try and find the cause.

 

 

 

 

 

 

 

This is what the player looks like inside, there is a power supply along the bottom left, a standard DVD ROM drive and a circuit board along the back.

The circuit board has a few obvious parts to it, at the top is a VIA chip, I initially thought this was a sound chip but a quick search for the part number VT6212L reveals it’s true purpose, a 4 port USB hub. This must be designed to act as an interface between the USB port on the back panel and the rest of the system because I can’t think of any other parts that run on the USB bus.

Next to this is a Realtek RTL8100C, this is the fast ethernet controller

Below this is a Mini-PCI Wireless LAN adapter which has the 2 antenna connectors attached to it.

Below this is a large chip, this is the video processor chip in this case a Sigma EM8622L.

Finally at the bottom is a Silicon Image SiI9030 chip which is the HDMI interface, this controls communications and encryption across the HDMI link.

Also on the board is a ST Electronics 64Mbit flash memory chip and 2 Nanya RAM chips.

As you can see from the image above there is no obvious damage, at least nothing that would cause the smoke I saw, this led me to believe the problem was in the DVD ROM drive. After spending 10 minutes fiddling round with it and eventually breaking several bits of plastic I managed to release the drive, when I removed the circuit board I found the following.

 

 

 

 

 

 

The chip had heated up and started to melt the plastic which was right above it, this could have been bad if this had happened when no one was around. The good news is the problem is hopefully contained to the drive which is a standard IDE DVD ROM drive and should be easily replaced.

I will post an update when the replacement drive has arrived.

Posted in Miscellaneous | Comments

Canon EOS 5D MkII

24 September 2008 – 4:30 pm

 

Canon recently announced the upcoming release of there new digital SLR, the 5D Mark 2.

This sounds like an amazing camera, it has 21.1MP sensor, a 3″ LCD screen with live view and an ISO that goes up to 25600. It also has another feature something Nikon recently announced on there D90, HD Video. I had initial thought recording video on an SLR would be a bad thing, by adding to many features you take away from what the camera is supposed to do but after seeing the results I have changed my mind.

Vincent Laforet has released a video for a mock trailer shot entirely using a Canon 5D, he had access to an early version of the camera so the video was ready for the release of the camera. The quality of the footage and the shots that were captured are amazing, and the best thing about having a camera like this is that you can easily and cheaply change lenses. I don’t know how much a comparable HD camcorder and lenses would be but I would expect them to cost a lot, probably more than the camera.

The one thing that is stopping me running out and buying this camera (apart from the fact it hasn’t been released yet) is the cost, the body is going to cost $2700 which will probably translate to £1500 in the UK, if you want to get a lens with it the kit will costs about £1800+.

Whilst this is a huge amount of money, especially for a student with photography being just one of many hobbies I am really tempted to get it. Perhaps I should invest in a decent lens for my current camera the now very old Canon 300D.

For an in depth review of the camera check out Thomas Hawk’s site.

Update: I have just read that the UK price for the body will be £2,300, and £3,049 for the kit; this is a ridiculous US to UK conversion and puts the camera completely out of my price range. I can only hope that it doesn’t stay at this price or I get a very well paid job.

Posted in Personal, Photography | Comments

Things to do

6 September 2008 – 9:57 am

I have come up with another thing for my list of things to do before I die; work for the British Antarctic Survey. The British Antarctic Survey or BAS is something I came across a few years ago, I am not really sure what it was about the type of job or experience that interested me but there was something. If your unaware of what the BAS is its Brittan’s research at the south pole, it includes several research bases covering different areas and different fields of science, you work there for periods of up to 33 months, that’s almost 3 years with limited contact with the outside world!

I am not a scientist so I wouldn’t go there to do research, my aim would be to get a job as one of the support staff possibly working with the computers, communications or electrical infrastructure. From reading there website the working days can be long especially during the summer when the most needs to be made of the available light but in your free time you can go and do thing like ice climbing which sounds really cool.

So that’s one more thing bringing my list to 4.

• Go into space / see the Earth from space
• Go sky diving
• Learn to fly a helicopter
• Work for the British Antarctic Survey

In terms of fulfilling number 3 (learn to fly a helicopter) I may have come a bit closer, I have found out you can buy a 30 minute flying lesson using Nectar points, something I am seriously considering doing.

Posted in Personal | Comments

Things to do before I die

1 September 2008 – 1:22 pm

Over the past couple of weeks I have come across a number of lists people have made identifying a number of targets people have set for there life, I quite like this idea so I have decided to create one of my own.

I think by having your ambitions, dreams, milestones or whatever you want to call it written down or at least well defined in your mind you will be more likely to achieve them. These aren’t things such as get a job or get married there much grander aims that can help shape your life by encouraging you to make decisions or take opportunities you may not have otherwise taken.

A little over a month ago I came across Randy Pausch when he sadly died of cancer, in amongst the tributes there were links to a lecture he gave at CMU titled the last lecture, there he gave this inspirational lecture during the last months of his life. The lecture focused on his life, covering his career, ambitions and achievements and is well worth watching. The point of mentioning it is that he started the lecture with a list of his childhood dreams, this is something I never had as a child or at any point in the 23 years until now, I am not sure why I guess I have never been much of a dreamer or looked towards the future like others have.

Over the weekend I started thinking of what would be on my list, because I don’t have childhood dreams to work on I have had to think of things I would love to do but also things that are ambitious, the first one I came up with was to go into space and to see the earth from space. On the surface this seems a little extreme but I genuinely think that I will be able to achieve this in my lifetime, in the next few years the first tourist flights to space will take place, just imagine how this industry will develop over the next 10, 20 or 30 years, hopefully well within my lifetime.

The other two things I came up with were to do go skydiving and to fly a helicopter, I think jumping out of an airplane and going in to free fall would be an incredible experience and something that shouldn’t be to hard to achieve. The third item is something I have wanted to do for several years now, I have thought for a while that being able to fly a helicopter would be a really cool and possible useful ability; I would love a helicopter license to put in my wallet with my driving license, thats assuming they come in a credit card size. This dream will probably be down to money rather then circumstance, but I will be on the lookout for any potential helicopter opportunities in the future.

These 3 things are all I have come up with so far, I am going to try and add to this list and hopefully use it as a guide of some kind for my life, there might not be much point to it but at least I will have an answer if anyone asks me what I want to do in life.

Posted in Miscellaneous, Personal | Comments

Photosynth - 3rd time lucky

24 August 2008 – 4:45 pm

I have tried a couple of photosynths inside today and I have got some better results. I photographed the lab I am working along with all of its mess, I took panoramas of the room in the centre and from various points around the room, I then took some detail shots of different things.

I have now got to synths of the same room on my profile because the system doesn’t allow you to edit existing work, what would be good is if it could detect similarities between your work and combine them.

The better one of the two is embedded below.

Posted in Internet, Photography | Comments

Photosynth - Attempt 1

24 August 2008 – 10:50 am

 
Yesterday I went to a National Trust property called Sissinghurst Castle Gardens, this is an amazing enclosed garden with hundreds of different type of plants where I was able to get a number of decent photos.

While I was there I took almost 200 photos for a Photosynth, I didn’t take the photos in the garden but behind the garden next to a lake. The Photosynth software seemed to do some pre processing of the images before they were uploaded, I don’t know to what extent the process was completed on my computer vs on Microsoft’s servers what I do know is the upload took hours and hours on my slow connection.

Today I am going to try another Photosynth but this time in doors, I will photograph my lab at university but hopefully in a more structured manor.

My Photosynth is embeded below

Posted in Internet, Photography | Comments

Data Security

22 August 2008 – 10:23 am

Yesterday there was yet another story in the news about the loss of sensitive data, this time records on all the prisoners in England and Wales as well as details on others with convictions were lost after a private company misplaced a memory stick. This is only the latest in a long line of data losses, earlier in the year CD’s and laptops containing details on others were lost as well.

This current issue raises several questions, first what was a consulting firm doing with information such as this? I can understand the need to view statistics and have information on prisoner distribution and the types of crimes committed but why wasn’t the data anonymized before it was given to this company and why wasn’t it encrypted.

There should be a simple policy for handling data such as this, never put it on removable media. Government agencies should have secure gateways, VPN’s or some other method that would allow those who are authorized to view this information without needing to copy it onto disk. Its going to take a serious incident (someone finding the lost data and doing something bad with it) before they really change there policies and way of working.

Posted in News | Comments

Photosynth is finally here

21 August 2008 – 11:42 am

A wile ago I came across Microsoft’s Photosynth, at the time I thought it was particularly strange; sometime after I came across it again and finally realized how amazing it was, since then I have been eagerly awating the release of the system to the public.

This morning I noticed several posts about Photosynth but I was in a rush so didn’t realize what had happened, it was when I got into work that I received the amazing news that Photosynth is finally here.

If you have no idea what Photosynth is I suggest you go an take a look although if you have a Mac you can read about it instead at Wikipedia, no Mac or Linux support I am afraid.

Rather than uploading the photos to Microsoft through the website you need to install a piece of software, I am unclear at this stage wether the software processes the images locally or acts as a facility for uploading them although I suspect it just uploads them, this then leads to the question of how long will Photosynth’s take to generate when they have been uploaded.

I will have the answer to all of these questions and more at the weekend, I was planning on spending the day taking photos and trying out my new tripod anyway now I have an even better task. Hopefully weather depending I will have my first Photosynth up at the weekend.

Posted in Internet, Photography | Comments