Posts Tagged AJAX



I’m scraping a site using curl (via PHP) and some information I want is a list of products which is by default only showing the first few ones. The rest is passed to the user when they click a button to get the full list of products, which triggers an ajax call to return that list.

Here is in a nutshell the JS they use:

headers['__RequestVerificationToken'] = token;
type: "post",
url: "/ajax/getProductList",
dataType: 'html',
data: JSON.stringify({ historyPageIndex: 1, displayPeriod: 0, productsType: All }),
contentType: 'application/json; charset=utf-8',
success: function (result) {
beforeSend: function (XMLHttpRequest) {
    if (headers['__RequestVerificationToken']) {
        XMLHttpRequest.setRequestHeader("__RequestVerificationToken", headers['__RequestVerificationToken']);

To set the request verification token as a header, more closely mimic an AJAX request, and set the content-type to JSON, use CURLOPT_HEADER.
curl_setopt($ch, CURLOPT_HTTPHEADER, array("X-Requested-With: XMLHttpRequest", "Content-Type: application/json; charset=utf-8", "__RequestVerificationToken: $token"));
PHP code
curl_setopt($ch, CURLOPT_USERAGENT, $userAgent);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_MAXREDIRS, 10);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookieLocation);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookieLocation);
curl_setopt($ch, CURLOPT_URL, '');
curl_setopt($ch, CURLOPT_REFERER, '');
$webpage = curl_exec($ch);
$productsType = trim(find_by_pattren($webpage, '<input id="productsType" name="productsType" type="hidden" value="(.*?)"'));
$token = trim(find_by_pattren($webpage, '<input name="__RequestVerificationToken" type="hidden" value="(.*?)"'));

$postVariables = json_encode(array('productsType' => $productsType,
'historyPageIndex' => 1,
'displayPeriod' => 0));
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array("X-Requested-With: XMLHttpRequest", "Content-Type: application/json; charset=utf-8", "__RequestVerificationToken: $token"));
curl_setopt($ch, CURLOPT_POSTFIELDS, $postVariables);
curl_setopt($ch, CURLOPT_URL, '');
curl_setopt($ch, CURLOPT_REFERER, '');
$webpage = curl_exec($ch);


Tags: , ,

JavaScript handling UTF8 characters

When I developed a PHP web application with AJAX. I encountered a problem that JavaScript cannot process utf-8 character correctly. Whenever user enter a UTF-8 string,
it was converted to ASCII in JavaScript and delivered to PHP script. The site is using UTF-8 characters and will be sure there are a lot of request using utf-8 characters. I googled “Javascript and utf-8” and find an article at It provides two functions for the purpose. One function I will use is the decode_utf8. I simply copied these two functions and the second one did not work. I change the inner function from escape to unescape. After the change, I call the function decode_utf8(s).

Here are the two functions:
function encode_utf8( s )
return unescape( encodeURIComponent( s ) );

function decode_utf8( s )
return decodeURIComponent( unescape( s ) );

A JavaScript code snippet to call the function:

function search(php)
var keyword = window.document.forms["searchform"].elements["keyword"].value;
var url=php+"?cmd=search" +
makeGETrequest2(url, link_callback);


Tags: , , , , ,

Tips for website development

I have encountered surprised issues when I developed Bookmark Manager 2.0. They were solved successfully after I spent a significant amount of time. To share my ideas and thought with friends, I summarized the processes and generated a list of a good practices as the following.

HTML/XHTML validation

It will always be a good practice to validate your webpages after the alpha or beta version of your website is built. If the XHTML format is adopted, the practice is more important. To do that there are a couple of choices:

I personally like the first one. It can provide a lot of detail and easy to understand. You should check your code to correct all the errors based on the validation output. After all errors are fixed, your pages are more stable than ever. When you adopt XHTML, the most common error is the close tag, for example, <br /&gt vs <br>, <script ….><script> vs <script …. />.

CSS validation

As the html page can be validated against standards, CSS also can be validated. It is always wise to validate your CSS styles and fix any errors or warnings. That will help your users and search engine bot. The website that provides service to validate CSS is:


Problem A: JavaScript does not run in IE and FireFox while it works fine in Google Chrome
Solution: Use encodeURL function to encode URL parameter in“GET”,url,true) statement, such as ““GET”,encodeURI(url),true);”.

Problem B: After the above changes, Opera, Safrari, IE, Chrome but FireFox are working properly. It is hard to find what was wrong in FireFox before I get FireBug extention. The problem is that FireFox cannot use the form name to refer the html element, such as “searchform.keyword.value”.
Solution: To solve the problem, have to use the “document” object to get it, for example, “window.document.forms[“searchform”].elements[“keyword”].value”. After the change, the JavaScript works fine for all major browsers: Chrome, IE, Sarafi, FireFox, and Opera.

Debugging Tools

FireBug is a great tool to debug the css, html, JavaScript and more.


Tags: , , , , , ,

A complete, generic AJAX framework (code provided)

AJAX technology is useful to update part of a web page. It is widely used on the Web 2.0 now. Several months ago I started to explore the AJAX technology and applied it to one of the web sites, U’s Bargain Network, that I managed. In order to use the JavaScript code across multiple website. I made it fairly generic. Since there are a lot of nice online tutorials available, such as the W3School AJAX tutorial, I do not want to explain the concept and technology and just provided a complete framework. You can modify to adapt to your specific situation.

// This following section is for AJAX activities
// Create by Zhanshan Dong, May 21,2009
// Modified on July 20, 2009
// Version 1.0.20

var xmlhttp;

// The following functions are for tags
var callback_links = function()
	if (xmlhttp2.readyState==4)

// example for GET request
function showLinks(id,admin)
	var url="showlinks.php"+
	makeGETrequest2(url, callback_links);

// example for POST request
function addLink()
	var poststr = "id_category=" + escape(encodeURI(addlinkform.id_category.value )) +
		    "&link=" + escape(encodeURI( )) +
		    "&description=" + escape(encodeURI(addlinkform.description.value )) +
		    "&name=" + escape(encodeURI( ));
	makePOSTRequest2('addalink.php', poststr, callback_links);

// common functions are for AJAX
// To make POST request
function makePOSTRequest(url, parameters, callback) {
	if (xmlhttp==null)
	  alert ("Your browser does not support AJAX!");

	xmlhttp.onreadystatechange=callback;'POST', url, true);
	xmlhttp.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
	xmlhttp.setRequestHeader("Content-length", parameters.length);
	xmlhttp.setRequestHeader("Connection", "close");

// To make GET request
function makeGETrequest(url, callback)
	if (xmlhttp==null)
	  alert ("Your browser does not support AJAX!");

// create xmlHTTPobject
function GetXmlHttpObject()
	if (window.XMLHttpRequest)
	  // code for IE7+, Firefox, Chrome, Opera, Safari
	  return new XMLHttpRequest();
	if (window.ActiveXObject)
	  // code for IE6, IE5
	  return new ActiveXObject("Microsoft.XMLHTTP");
	return null;

If you want to update two area at one call, you can create a second xmlHTTPobject for this purpose. There are many ways to use the AJAX framework and functions. Here are several suggestions.

  • Call it at page loading stage
    <body onload="javascript:showLinks('36','0');">
  • A majority of time, the AJAX functions can be called from form submission
    <form name="addcatform" method="post" action="javascript:addLink();">

If you want to update multiple areas in a AJAX function call, you have to define a second xmlHTTPobject and then call multiple AJAX functions like the following:

onload="showComment(12401); showTags();"

Enjoy the generic, complete AJAX template and develop your Web 2.0 website.


Tags: , , , ,