Django

6 10 2009

I mentioned the Django app I made in the previous post so I thought I would provide some info about what it is. Essentially the goal was to reduce duplicate work everywhere possible be moving data from spreadsheets and other database’s into one central database. Also to allow a non skilled worker to edit this data, then allow for reporting to remake all the excel sheets that were originally needed. Here’s my setup

Ubuntu 9.04 server running Django, MySQL, and Apache (LAMD?) Data models are defined in Django which automatically makes an Admin interface with some customization options.

data entry

Django also makes short work of authentication with is done against Active Directory. Reporting is done with PyExcelerator and pyRTF to make downloadable excel and rtf documents.

reporting

It’s a pretty basic database but it really saves a lot of time compared to maintaining lots of xls documents and mail merges. Also it allows a technical worker to import exported data from other database into MySQL. Ideally this program could also be linked with other MySQL backend programs. So say I want to use SugarCRM I could symlink the contacts table so both Sugar and Django use the same one for perfect 2-way syncing. The real beauty of this is that it was so quick to develop. This is just been a side project for me. Doing it in PHP or .NET would have easily taken 3 times as long.





Working away

30 09 2009

Well I’ve been in nyc for a month now. The first few weeks have been hell with 60+ hour weeks at Cristo Rey but things are finally starting to calm down. At work I’m coordinating transportation for students to get to their work placements.

I’ve completely redid the process in a short time I was there from a bunch of random Excel files and proprietary databases to something more maintainable, a MySQL database with Django. I’m impressed with Django’s ability allow me to make good data centric websites in only a few days. Django’s philosophy of defining data “models” once and having it create the database and administration page automatically is great. I’m then using pyRTF and pyExcelerator to generate reports from the data. We can now enter student, company, and contact data in at one place and have it reflect to all relevant reports such as daily attendance. The admin interface is easy enough to use that students can do data entry with it.

Other new fronts include the possibility of moving from Act by Sage to SugarCRM should further streamline the process. The idea here would be that Sugar has more features and could integrate with my Django database, Outlook, and a smart phone. With some hacking around it looks like I can symlink(yay unix) a “contacts” table used in both Django and SugarCRM to keep them perfectly synced and keep Django happy in it’s data model land without manual SQL needed. I’m happy to be using my skills at the new placement, while also running the day to day activities at the school. Though it’s still a 10+ hour day with some Saturdays making it rather stressful.

Other thing’s I’m looking into are Alfresco content management system, Zimba email server, and SchoolTool. SchoolTool is a decent school administration management tool. It’s written in Zope which is a python based framework. Python is quickly becoming my favorite language. It’s missing a few key features so I might hack on it to make it work. One unsolvable(?) problem with SchoolTool is that it uses ZODB, an object oriented database. This means it would be really hard to integrate it with the other databases I’m using. ln -s can’t save me this time..





Volunteering

24 08 2009

The following was written on 8/16/09 though I did not have Internet access at the time.

I am at the Jesuit Volunteer Corps (JVC) retreat house; we are doing a personal time session and I have decided to write this entry for my blog. I am the only engineering and the only IT person here. The complete lack of relevance to my chosen profession makes me wonder what I am doing here, as do the strange bugs that roam the area at all times. I am also in the midst of like minded (go social justice!) individuals. Something unexpected are the type of people here. I am surprised by the number of frat people who have come here, which is maybe from my own ignorance. I will be placed into the only all male JV house, which to me seems strange since I have lived often with girls and was expecting it here. As always I am both in and out of my element.

In a few weeks I will start working at the Cristo Rey High School in NYC. After being informed I would be taking care of filing there, I immediately determined in my own head I would independently digitize their records keeping system. We’ll see if that happens. I am already leaning the Django, a python based web development language. Django attempts to remove duplicate and inelegant work from web site construction. Data objects and classes are defined once in a model and used in a database, front end view, and automatically creates an admin view. This will extend a site’s ability to be maintainable by even less than technical people, as a web interface can let them add or change content. Since I’ll be working past full time and be kept busy with community nights, I’ll need to develop things very quickly. Also I always pick a different language for every project I do. I am excited to start, and probably their only volunteer anxiously waiting to know what servers they use.





Thinking about business

5 08 2009

I’ve been thinking about starting a business after I get a few more years of work experience. Some type of computer retail and software consulting thing. Here’s some random things that might be fun and profitable that could be thrown together under one roof.

  • Sell custom computers without bloatware. Linux, Windows, even Hackintoshes. Preload software that is actually useful. Instead of Microsoft Works, use OpenOffice. Instead of some trial for anti virus for Windows, ClamWin or AVG. Have different versions ie (Student, easy enough for grandma, hAx0R 1337 teenager),
  • Allow anyone to use the computers on display, as long as they want. Sell some coffee and pastries to them (think Barnes and Noble with Starbucks) When they see our $300 linux box outperforms their $700 spyware ridden and bloated Dell they will want one.
  • Lease option. Offer to manage computer remotely. Many people might just want a Internet or Word processing appliance. This lets them forget about all the computery things like updates.
  • Lauch computer donation project. Recruit volunteers to do most of the work. Could generate some buzz amongst the geeks in the community, a major target market.
  • Develop software for anything. Try phone apps, websites, consulting, email, web hosting. Just see what works.

We’ll see where this goes.





Evil Evil Video

10 07 2009

My work deals with videos. I take them from DV tapes to viewable, downloadable video on a website. In fantasy land I can just copy video from a camcorder, upload it, and I’m done right? Nope! HTLM 5 attempts to solve the problem with the new video tag. But some people (Apple) want to use various MPEG files like H.264 for HTML 5 Video. Well that isn’t practical. Yes H.264 is a good format. H.264/MPEG-4 have patents and users must pay a fee to use them. Distributing them with say Firefox, isn’t possible. Second lets say someone paid a lot of money to allow it’s distribution. I can’t redistribute software that encodes MPEG4 for free (legally). So my client’s can’t transcode their videos. Do I tell them oh just send a check too…yea right.

There is a solution hack
Ogg Theora It handles video great and has a quality to disk space ratio competitive to H.264. It also handles shaky camcorders better which is huge for what I do. I can easily make a script with ffmpeg2theora that converts dv to ogg theora. I use virtualdub for extracting the DV tape video and avisynth for editing if anyone is curious. Now it’s easy to convert a large avi into a much smaller ogv file ready for uploading.

Next the server runs cron jobs to encode it as flv flash and MPEG1. MPEG1 is sometimes requested by the same companies that run Internet Explorer 6 with no plans up updating it ever. Flash if for backup since most people won’t have a browser than can play HTML 5 video. Since the flash video in my case is only for previews really, I can drop it’s quality big time so the flv file doesn’t chew through disk space. MPEG1 quality is dropped too because MPEG is horrible. Only a few clients need it so tough they get low quality. I play the flash with FlowPlayer and play the video with something like this
<video src="file.ogv" >
Play flash with flowplayer if html5 video fails.
</video>

Now if fully digital camcorders would get cheap, we could stop using those intensely annoying DV tapes.





Teaching programming part 2

28 06 2009

Ok here’s my favorite program to play with for teaching children their first lesson
// Simulate the world being destroyed (or prospering)
using System;
namespace worldsim
{
class MainClass
{
public static void Main(string[] args)
{
// Number of life forms on Earth
int people = 400;
int animals = 300;
int plants = 500;

int day = 0; // Days past since start of simulation
int maxDay = 100; //

// while days not maxed and there are more than 0 people
while(day 0)
{
people = Convert.ToInt32(Convert.ToDouble(people) * 1.02);
people--;

animals = Convert.ToInt32(Convert.ToDouble(animals) - (Convert.ToDouble(people) * 0.1));
animals--;

if (animals < 0)
{
animals = 0;
// Starving people
people = Convert.ToInt32(Convert.ToDouble(people) * .9);
}

day++;


// Show daily populations report
Console.WriteLine("day: " + day);
Console.WriteLine("people: " + people);
Console.WriteLine("animals: " + animals);
Console.WriteLine("plants: " + plants);
Console.WriteLine();
}
}
}
}

I just give this code to everyone after briefly explaining a few basics of programming, introducing them to a terminal, etc. Programming in crazy. You can’t just learn it quickly through lecture; you learn by doing. This program is a primitive world simulation where humans and other lifeforms exists, eat each other, and reproduce. By a few lines of code we can make the population explode, die out, or attempt to reach equilibrium. The Convert.Towhatever I say is evil magic for now and just copy and paste it as needed. Same with using System; The ; is evil magic too, designed to make programming hard. The point is to have fun. You know its working when kids start coding in lines that contain words they think you would yell at them for writing. The best part is you see how each student learns almost instantly! Some will do exactly what you tell them and try to make a persistent where all the creatures co exist. Some will work together while some will turn to you for help. Now you can cater to each type. The students goofing around thinking they are 1337 haX0rs making flying pizza’s attacking digital humans while want more challenges and must be allowed for creativity, else they will start playing video games. Others need some hand holding and more explicit direction before they get too deep into all this crazy programming jibberish. Either way is fine.

This program also makes a great lead-way to learn object oriented programming, since it’s easy to make people into objects with methods like eat, reproduce, etc. But that’s later. Now they need to know how to program and what the “evil magic” is. This stuff is boring at first, but hopefully after seeing what programming can do they will be motivated for the task. Also I stay away from integrated development environments at least for a lesson or two. They teach kids to follow the evil magic of pressing buttons and suddenly the code works. This leads to treating code like witchcraft. They might start putting ;’s all over the place just to make it work, instead of figuring out why the ; should be there in the first place. That’s a bad habit that some people never break out of. They need to know what a complier, managed vs unmanaged code, and libraries are. Lectures about these topics are garbage if they know pressing F5 makes the program magically work, they will ignore you. Sure you could test them on knowing these buzzwords, but then they memorize instead of comprehend. So get in a terminal and compile! Make sure all the evil magic of programming is turned into understandable ideas.

Oh and one more thing, please don’t use the boring Lets Make a Calculator! program. Try to get feedback from kids and let them decide what they want to make. They are learning concepts, not how to make a stupid program that’s already been made 100,000 times before.





Teaching Programming Part 1

15 06 2009

In my experience people generally teach programming poorly. A paper I wrote about this “Incorporating Gaming in Software Engineering Projects: Case of RMU Monopoly” will be in the next issue of http://www.iiisci.org/journal/sci/Home.asp

Summary – programming should be hands on and students should choose how to work, what to work on, and what tools to use. Instead students memorize how to write classes and learn about something they call objects, but don’t know why. Syntax is treated like witchcraft to make the program work and get an A. Also many professors are just boring and questionably interested in programming themselves. Others are arrogant and unapproachable. A good programming class lets students decide what they want to build. Lectures go over concepts and students then implement them how they see fit. Of course the instructor needs to be there to help them.

Some instructors seem to think students shouldn’t be allowed to look on the Internet for answers. That just crazy. At least 50% of my time at work is spent looking around on Google. Of course students in their first programming class may need a simple program to start with. Next week I’ll talk about how I teach programming.