Archive for category XML

Use PERL script to automatically follow your followers on Twitter

If you have a Twitter account, you will know there are friends follow you and you are not following them. To be nice to your fans, you’d better follow them back. If you have a large number fans added everyday, it will be hard to catch up. Now it is time to automate the process. The following PERL function will do the trick for you.

#!/usr/bin/perl
require 5.6.0;
use strict;
use warnings;
use DBI;

# ==== get command line arguments
if ($#ARGV != 1) {
    print “usage: twitter_friends2 username password\n”;
    exit;
}

my $username = $ARGV[0];
my $password = $ARGV[1];

my $dbh;
my $sql;

getConnected();

my $friends = “friends”;
my $followers = “followers”;
my $target_followers = “target_followers”;

# ==== download friends, followers and target’s followers list
system(”curl -u $username:$password http://twitter.com/friends/ids/$username.xml >$friends.xml”);
system(”curl -u $username:$password http://twitter.com/followers/ids/$username.xml >$followers.xml”);

# ==== clean data tables
$sql =”DELETE FROM friends”;
executesql($sql);
$sql =”DELETE FROM followers”;
executesql($sql);
$sql =”DELETE FROM nonfollowers”;
executesql($sql);

# ==== load data to tables
importIDs($friends);
importIDs($followers);

# ==== find non following
$sql = “INSERT INTO nonfollowers (id_nonfollowers) “. 
 ”SELECT id_followers FROM followers WHERE id_followers NOT IN “. 
 ”(SELECT id_friends FROM friends)”;
executesql($sql);

# follow the friends you are not following now
add();

disConnected();

# END OF MAIN PROGRAM

#############################################
#
# add friends who do not follow you in the past
#
sub add {
my $sql = “SELECT * FROM nonfollowers”;
my $sth = $dbh->prepare($sql);
$sth->execute();
my $id;
$sth->bind_columns(\$id);
while ($sth->fetch()) {
   system(”curl -X POST -u $username:$password http://twitter.com/friendships/create/$id.xml >/dev/null”);
   my $sql1 = “DELETE FROM nonfollowers where id_nonfollowers=$id”;
   my $sth1 = $dbh->prepare($sql1);
   $sth1->execute();
   $sth1->finish();
  }
$sth->finish();
}

#############################################
#
# remove friends who do not follow you
#
sub remove2 {
my $sql = “SELECT * FROM friends”;
my $sth = $dbh->prepare($sql);
$sth->execute();
my $id;
$sth->bind_columns(\$id);
while ($sth->fetch()) {
   system(”curl -X POST -u $username:$password http://twitter.com/friendships/destroy/$id.xml >/dev/null”); 
   my $sql1 = “DELETE FROM friends where id_friends=$id”;
   my $sth1 = $dbh->prepare($sql1);
   $sth1->execute();
   $sth1->finish();
   }
$sth->finish();
}

#############################################
#
# execute a SQL statement
#
sub executesql {
my $sth = $dbh->prepare($_[0]);
$sth->execute();
$sth->finish();
}

#############################################
#
# import data to table
#
sub importIDs {
my $tablename = $_[0];
open(DAT, “$tablename.xml”) || die(”Could not open file!”);
my @lines=;
close(DAT);

my $sth;
my $nIDs = 0;
foreach (@lines) {
my $id = $_;
$id =~ m/([\d]*)<\/id>/;
if (defined($1)) {
   my $sql = “INSERT INTO $tablename (id_$tablename) values ($1);”;
   $sth = $dbh->prepare($sql);
   $sth->execute();
   $nIDs ++;
   }
}
$sth->finish();
unlink “$tablename.xml”;
return $nIDs;
}

#############################################
#
# connect to database and generate a handle
#
sub getConnected {
# return the database handle object to the caller
# Set the parameter values for the connection
#——————————-
my $host=”localhost”;
my $connectionInfo = “DBI:mysql:db_twitter;$host”;
my $databaseUser = “your database username”;
my $databasePw = “your database password”;

# Connect to the database
# Note this connection can be used to 
# execute more than one statement
# on any number of tables in the database
#——————————-

$dbh = DBI->connect($connectionInfo, $databaseUser, 
   $databasePw) || die “Connect failed: $DBI::errstr\n”;    
}

######################################
#
# disconnect the database handle
#
sub disConnected {
   $dbh->disconnect();
}

To use this PERL script, you have to create a database in MySQL server by using the following SQL commands.

/*
DATABASE db_twitter
*/
/* 
followers
*/
CREATE TABLE `followers` (
  `id_followers` int(15) unsigned NOT NULL,
  PRIMARY KEY  (`id_followers`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
/* 
friends
*/
CREATE TABLE `friends` (
  `id_friends` int(15) unsigned NOT NULL,
  PRIMARY KEY  (`id_friends`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
/* 
nonfollowers
*/
CREATE TABLE `nonfollowers` (
  `id_nonfollowers` int(15) unsigned NOT NULL,
  PRIMARY KEY  (`id_nonfollowers`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;

Once you ccreate the database (db_twiiter), you can run the PERL script to automate follow your friends.

Reference

Share

Tags: , , , ,

An example of processing XML file – treat it as text

The following example describes a way to process XML file, treating the XML file as text file. We all know that XML file is well structured and can be read and written by using specilized functions and classes. Sometime it is more convenient to treat the file as text file. Here is an simple example.

We have a series of data rows stored in a spreadsheet. The XML file includes a template that tell us how we can use the stored data to generate new items. Stored data only covers part of attributes described in the template. We’d like replace attrabutes in template with the available and leave the others as they are. It is easy to write a VBA subroutine to fullfil this purpose without using any XML processing functions.

The first row in the spread sheet includes name of each attribute. From the second row on, each row represent a data set for generating new items. Some cells in the row might be empty. The provided VBA code includes the mechanism to deal with the empty cells.
The VBA code includes three subroutines. The first one is main processing. The remaining two deal with reading and writing XML files. So the logic is very simple.

VBA Code
========
Option Explicit

Private temstr As String
Private lines

Sub insert_data()

    readxmltext

    lines = Split(temstr, Chr(13))

    Dim attr_name(14)
    Dim attr_pos(14)
    Dim attr_str(14)

    ' get attribute name
    Dim r, c, i, j, k
    r = 1
    For c = 2 To 15
        attr_name(c - 2) = ActiveSheet.Cells(r, c).Value
        attr_pos(c - 2) = -1
    Next c

    ' find attribute positions in temstr
    For i = 0 To UBound(lines)
        For j = 0 To 13
            Dim pos
            pos = InStr(lines(i), attr_name(j))
            If pos > 0 Then
                If (attr_pos(j) < 0) Then attr_pos(j) = i
                Exit For
            End If
        Next j
    Next i

    ' generate new items
    For r = 2 To 23
        For c = 2 To 15
               Dim attr_value
               attr_value = ActiveSheet.Cells(r, c).Value
               attr_str(c - 2) = "<" & attr_name(c - 2) & _
                  ">" & attr_value & ">"
               If (attr_value <> "") Then
                  lines(attr_pos(c - 2)) = attr_str(c - 2)
               End if
        Next c
        writexmltext
    Next r
End Sub

Sub readxmltext()
    Const ForReading = 1, ForWriting = 2, ForAppending = 3
    Dim fs, f
    Set fs = CreateObject("Scripting.FileSystemObject")
    Set f = fs.OpenTextFile("template.xml", ForReading)
    temstr = f.ReadAll()
    f.Close
End Sub

Sub writexmltext()
    Dim fs, f
    Set fs = CreateObject("Scripting.FileSystemObject")
    Set f = fs.OpenTextFile("new.xml", 8, True)
    Dim i
    For i = 0 To UBound(lines)
        f.Write lines(i)
    Next i
    f.Close
End Sub
Share

Tags: , ,

Hands-on Instruction for Installing Ubuntu Server Edition to EEE Box

Introduction

EEE Box (buy a new one with HDMI output only $219.99 at buy.com with this coupon) is a nice min desktop machine, quiet and fast enough for a home file and web server. Since there is a need to create a web server and mysql server, I bought a coupple of EEE Box recently. One has Windows XP as OS and the other with Linux as OS. I want to install Debian or Ubuntu Linux Server to the machines. First thing I did was to search how to install Ubuntu to the machines. After I read articles in several forums, I had rough idea how to do it. In one weekend, I started to follow information in one of the forum articles and ended up no success.

The first strange thing I found is the boot sequence of EEE box. I changed the sequence of bootable drives in “Boot Setup” -> “Boot Device Priority”. It did not have any effect when I rebooted the machine with a USB memory stick. Then I went back to the Boot Setup menu again. I saw there is one more choice – “Hard Disk Drivers”. In this menu, I could change the 1st drive to USB memory stick. Once the change was saved and the machine is rebooted, it actually used the USB memory stick to boot the machine. But there is no valid OS or boot information in the memory stick. I had to search internet again and to find more information to prepare USB memory stick. That ended up the following successful story, a hands-on step by step instruction to install Ubuntu Server Edition to EEE Box machines.

Prepare a USB memory stick

First things first, we need download UNetbootin. This is handy software working both in Windows and Linux envrionments. Here is its offical description: “UNetbootin you to create bootable Live USB drives for a variety of Linux distributions from Windows or Linux, without requiring you to burn a CD. You can either let it download one of the many distributions supported out-of-the-box for you, or supply your own Linux .iso file if you’ve already downloaded one or your preferred distribution isn’t on the list.” It can handle all kinds of Linux system, such as Ubuntu, Debian, Gento and so on. To visit the website to learn more about it.

Second, we need download Ubuntu Server Edition, the iso file for installation. Be sure to click the server edition icon in the page and download the right iso file. I personnally prefer to bring everything locally and mke sure the installation go flawlessly and smoothly.

Once you download both files to your computer, you can start UNetbootin program and select install from ISO file and choose your USB memory stick. Installing the system the memory stick is a piece of cake.

Handle BIOS setup

EEE Box boot setting is little bit different from other machines. Be sure to follow the instruction here to change the boot device priority correctly.

  1. Please insert the USB memory stick you prepared in the first step before you reboot your system
  2. Press [DEL] key after EEE Box splash screen, it brings you to BIOS setup program
  3. Use you cursor to navigate to “Boot” menu. “Boot Settings” is under this menu
  4. Select “Hard Disk Drivers” to progress to next screen, which lists all available “hard drivers” for selection
  5. Highlight “1st Drive” and change the selection to “USB:Kingston DataT” (in my situation, yours will be different)
  6. Press [Esc] key back to the parent menu and then press [F10] key to save the changes you just did and rebout the system

Install Ubuntu Server Edition to EEE BOX

After your system reboots from the USB memory stick, a menu will pop up. There are a number of choices available. Select “Install” in the startup menu to start install Ubuntu system to your hard drive. Here I will not expalin the detail installation. You should follow the screen prompts to choose appropriate options to install your server OS or refer to proper sections in Ubuntu website.

Install GRUB to hard drive

If your system had Windows XP running before you installed the Ubuntu system, you need complete the following stem, otherwise you will get error message during reboot. The stupid installation program did not know writing GRUB bootloader to the hard drive you were working with. In stead, it wrote GRUB to your USB memory stick and left your hard drive unbootable.

  1. Boot your system with the USB stick in one USB port
  2. Make your hard driver bootable by the following command# fdisk /dev/sda
    type “a” to change the first partition (1) to bootable
    type “w” to write change to hard drive
  3. Install GRUB to your hard drive# cd /usr/sbin/
    # ./grub-install /dev/sdc

“/dev/sdc” in the above command is your hard drive. You might need change it based on your machine. Once these steps are completed, you can reboot your computer without USB stick normally.

First reboot

If you did not choose installing GUI interface, you will see a lot of information in text mode, which tells you what devices are started correctly, what modules are started and so on. Shortly the log in prompt will show up. You can use the admin user and password you created during installation process to log in the brand new Ubuntu system. Congratulations!

Based on Ubuntu description, the root accout is disabled at default since the admin account has full power to work with the system. By any reason you want to enable root account, please following the simple step below to do it.

# sudo passwd root

Enter your admin user password
Enter new root password
Reenter new root password

Once you finish the above command successfully, your root account is enabled. To log in root account you can use the su command.

# su
Entry root password

Sweet. We are in root account. Remember that you have unlimited power to change anything in the system. So be careful when you do anything for the system. From now on, you can enjoy the robust Linux server serving for you quietly and efficiently. The can run months without any maintenance.

This page you are reading is served by the lovely machine with Ubuntu Server Edition OS.

Share

Tags: , , , , , , ,