Posts Tagged ‘Data’
Thursday, October 30th, 2008
UPS. UPS in this context does not mean United Parcel Service, it means Uninterruptible Power Supply. This is a battery that sits between the wall socket and your computer. If there is a power outage or brownout, the UPS instantly kicks in and keeps your system from crashing.
Depending on the size (and cost) of the UPS, it can keep your system running for a few minutes or for several hours. A large hospital will have huge (and expensive) UPS’s that will keep their computers up for as long as it takes to fire up their backup power generators! This is something that I would appreciate if I were having open-heart surgery at the moment of a blackout, but it is probably beyond the budget of the small business owner or home PC user.
For those of us on a budget, a small UPS may be the best insurance policy we could buy. A small UPS will keep your system running for 5-10 minutes. The purpose is not for you to keep working (as it would in the hospital scenario,) the purpose is to give you enough time to save your work and perform an orderly shutdown. It could save you a fortune in lost data. Another benefit of a UPS is that some of them (the good ones) act as a super surge protector. At least one on the market is warranted to stop a direct lightning strike! It will self-destruct in the process, but it will stop the surge from hitting your PC. That’s a lot of protection for about $100.
A wise old computer guru once told me that the value of a PC is not in the cost of the hardware, the value of a PC is in the data that you store on it. You can always write a check for new hardware, but what will it cost you if you lose your data? Could it be replaced for ANY amount of money? If you depend on a PC for your livelihood, a UPS can be the single best hardware investment you can make.
Tags: backup, battery, Computer, Data, hardware, PC, UPS
Posted in news | No Comments »
Thursday, July 24th, 2008
Often you have a table in which only a few columns are accessed frequently. On a blog, for example, one might display entry titles in many places (e.g., a list of recent posts) but only ever display teasers or the full post bodies once on a given page. Horizontal vertical partitioning helps:
CREATE TABLE posts (
id int UNSIGNED NOT NULL AUTO_INCREMENT,
author_id int UNSIGNED NOT NULL,
title varchar(128),
created timestamp NOT NULL,
PRIMARY KEY(id)
);
CREATE TABLE posts_data (
post_id int UNSIGNED NOT NULL,
teaser text,
body text,
PRIMARY KEY(post_id)
);
The above represents a situation where one is optimizing for reading. Frequently accessed data is kept in one table while infrequently accessed data is kept in another. Since the data is now partitioned the infrequently access data takes up less memory. You can also optimize for writing: frequently changed data can be kept in one table, while infrequently changed data can be kept in another. This allows more efficient caching since MySQL no longer needs to expire the cache for data which probably hasn’t changed.
Tags: accessed, blog, changed, Create, Data, Frequently, infrequently, MySQL, needs, Optimizing, Partition, probably, reading, Table, Tables, your
Posted in MySQL, tricks | No Comments »
Thursday, July 24th, 2008
Frequently we query our database for a value but then have to always do the same manipulation on it before we can use it. Well it can often be useful to do this “once” in the database and then store the manipulated value in a separate column of the database.
Here is a very simple example
INSERT INTO tbl_name (col1,col2) VALUES(15,col1*3.14159);
Where col2 is set to the value of col1*pi
You could also do more complex calculation/filtering/shortening in your code eg php,ASP etc
Tags: column, Data, database, Derived, Frequently, insert, INTO, manipulated, MySQL, once, separate, useful, values
Posted in MySQL, tricks | No Comments »
Thursday, July 24th, 2008
Columns with identical information in different tables should be declared to have identical data types so that joins based on the corresponding columns will be faster.
Tags: corresponding, Data, different, faster, MySQL, should, Synchronize, types
Posted in MySQL, tricks | No Comments »
Thursday, July 24th, 2008
Use insert delayed when you do not need to know when your data is written. This reduces the overall insertion impact because many rows can be written with a single disk write.
Tags: because, Data, delayed, Disk, Impact, insert, MySQL, overall, written
Posted in MySQL, tricks | 1 Comment »
Tuesday, July 15th, 2008
Google Analytics is a great program that can do a lot more than most people realize. Here are a few features that you may not know about:
* Capture internal search stats. This is a newer feature of Google Analytics and a very nice one. Not only can you see what a visitor put into your internal search box, but what page they were on when they made the search and what page they chose in the search results. Any internal search will work as long as it passes the search variable through the URL. Here’s a short video interview with Google’s Brett Crosby on some of those features.
* Filter out domains. Let’s say your Google Analytics code somehow got on another site and your stats were getting tainted with irrelevant data. No problem. You can create a filter to not count anyone from specific domains you add in. Oddly enough, you can also filter out your own domain so your stats flat line. Not a good idea to do that though.
* Track document downloads or specific links. Adding a small piece of JavaScript to any link will tell Google to track when someone clicks on that link. This works for PDFs, Word documents, email address’ and external links. It also works if you want to see which two links on the same page are generating more clicks. Even though they go to the same URL, you can tag one link as ‘link one’ and the other as ‘link two’ and Google will track the clicks separately for you. Bonus Tip: If you have a Wordpress blog, you can instantly tag all links across your blog with the Ultimate Google Analytics plugin.
* Export to Excel. For any newbies, this is a time saving feature. Just about any report can have the data exported to a CSV file which Excel can open. You can now stop copying and pasting most data out of Google Analytics and into Excel and save yourself some time.
* Filter yourself out. This feature is a must do for any company. Find your static IP address and then set up a filter so Google knows not to include traffic from your company network. This ensures that your stats are not inflated due to employee’s surfing habits. This also is something you should consider for any partner companies. If you have a web development, or SEO company who is constantly checking out the site, filter them out too.
Tags: capture, company, company network, Data, downloads, Excel, features, few features, Five, Google Analytics, IP address, Javascript, Known, PDFs, people realize, plugin, program, SEO company, web development, WordPress, yourself
Posted in google, tricks | No Comments »
Tuesday, July 15th, 2008
My ‘day job’ has nothing to do with PHP. It has nothing to do with any form of programming. I graduated in 2006 with a degree in Biochemistry and went on to do a MSc and now PhD in cardiovascular biology. The closest most of my colleagues come to programming is a formula in an Excel spreadsheet.
It was actually Excel which prompted this post. Yesterday I was analysing some data and bemoaning the poor search functionality that Excel makes available. I had already expanded the small set of experimental data I had with some values pulled from a web service using a quickly hacked together PHP script and it got me to wondering how much better things could be if I just stuck with PHP.
Where’s the science?
This train of thought led on to whether PHP has been used all that often for scientific projects. There is an accelerating trend in Biology to make data and tools available via web interfaces. In my opinion this is an environment where PHP excels and yet all the literature I’ve seen discussing the development of these services uses Perl or occasionally Java.
Searching a little harder for PHP projects yields an equally depressing outlook. In PEAR Jesus Castagnetto released the Science_Chemistry and Math_Stats packages back in 2003. For my purposes though the Chemistry package is a little too ‘chemical’ and the stats package is a little too basic. In sourceforge there is a package named BioPHP which looks promising but again there has been no activity since 2003. A lot has happened since then.
Biology is increasingly data generative. There is going to be a steadily increasing need for tools to analyse all this data. These are likely to be centralised and made available via web interfaces.
Anyone out there?
I suspect I’m going to be increasingly creating automated solutions to remove some of the repetition involved in processing the, relatively, small amounts of data that I generate. A PHP toolkit able to leverage the latest online databases and perform ‘advanced’ statistics would be immensely valuable.
So my question is this. Is anyone out there using PHP in a scientific environment? Are there resources available which I’ve missed?
Tags: actually Excel, analysing, available, bemoaning, colleagues, Data, day job, depressing, development, enough, Excel, expanded, experimental, formula, functionality, good, graduated, Jesus, PHP, Programming, prompted, science, Script, small set, spreadsheet, wondering
Posted in SEO, google | No Comments »
Monday, June 30th, 2008
I came to a point where I needed to filter the user input (not just tell the user something like : “The username isn’t in the right format” or “Please choose a valid name for your username”), I mean let him to input in the form anything he wanted too but in the end what we store in the database is his processed input (only A-Z, a-z and 0-9).
I had the Visual Basic and C++ thinking .. I must do that and that and that to get rid of unwanted characters. It was a little time consuming and I was in hurry so I googled and I reached the ereg_replace help page on PHP.net website. Suddenly all became much much easier :D, just write a regular expression (ohh, by the way .. if someone asks I can write a small tutorial on regular expressions) like this : “[^A-Za-z0-9]” and that was it :).
Below is a small example on how to filter (remove all characters except uppercase/lowercase letters and numbers)
1. <?php
2. //our string
3. $str = “nek#$#hbet|20&&07″;
4. $filtered = ereg_replace(”[^A-Za-z0-9]“, “”, $str);
5. print $filtered; // will print ‘nekhbet2007′
6. ?>
PHP is a strong language with a lot of functions that help us to decrease the development time. So before thinking how to do a certain thing just take a look first at www.php.net and see if what you need isn’t already done
Tags: anything, C, characters, Data, database, functions, googled, name, PHP, Protect, user input, username, valid, your
Posted in PHP | No Comments »
Saturday, June 28th, 2008
Certain characters, for example ‘&’, have special significance in HTML, and should be represented by HTML entities if they are to preserve their meanings. There are several stages for which encoding is important. Assuming that you have a string $data, which contains the string you want to pass on in a non-encoded way, these are the relevant stages.
Passing a value through HTML FORM you must include it in double quotes, and htmlspecialchars() the whole value. For example see the code below
<?php echo “<input name=’data’ type=’hidden’ value=’” . c($data) . “‘>”; ?>
While passing a value through URL you must encode it with urlencode(). It will convert all non-alphanumeric characters except -_. with a percent (%) sign followed by two hex digits and spaces encoded as plus (+) signs. See example below.
<?php echo “<a href=’” . htmlspecialchars(”/nextpage.php?stage=23&data=” .urlencode($data) . “‘>\n”; ?>
Tags: Data, decoding, echo, encode, encoded, Encoding, htmlspecialchars, input, non-alphanumeric, passing, URL
Posted in PHP | No Comments »
Thursday, June 26th, 2008
Most web hosting packages include one or more databases. What can you use them for? How will they help your website? Read on for the answers to these questions. A database stores data, but more importantly, allows that data can easily be accessed. Data can be product information, customer names and addresses, sales records, or even the information that appears on web pages. Using a database to retrieve that information can allow you to better serve your visitors and provide them with a more interactive experience.
The most common use of databases in an Internet environment is to serve information dynamically as it is requested. In a large eCommerce site, for example, the actual product information is maintained in a database so that updating the site is a simple matter of changing the data. Without this system, website managers would have to create static pages for each product. When dealing with hundreds or thousands of products, this task would be almost impossible to manage in an efficient manner.
Dynamic pages use a template for the static content of the site such as headers, menus and footers. The contents of the database are inserted into the template by the server software before the page is sent to be viewed in a browser. Any content from the database can be placed anywhere on a dynamic page. This allows you to set up visually appealing pages which include text and pictures and also add shopping suggestions like: ‘Customers who bought this also bought…’
Databases can also be used for storing and accessing customer records. This allows you to tailor your pages according to your customer’s previous purchases. Each page could have a personalized greeting (Welcome back Peter) and when they make another purchase all their personal data including address and credit card number could be pulled from the database so they don’t have to fill in the same form again.
Another use of a database is mailinglists. Many websites send out information to their visitors to remind them about the site and encourage them to visit again. Email addresses can be stored in a database for the purpose of sending out announcements and newsletters. Also the newsletters can be archived in a database so that visitors can browse or search through previous mailings.
Each database can be divided into tables which are a complete set of data, so one database could be used for most of your website information by setting up a number of tables.The number of databases that your site needs depends on how many applications you are going to run.
It’s one thing to have a database, it’s another to access that data. There are several ways to retrieve information from a database so that it can be usefully applied to your website. One of the most popular combinations is PHP along with MySQL. PHP can be used to create dynamic web pages that pull data from a MySQL database. The programming language is quite straightforward and can be used to set up complex interactive forms. Other database applications include MySQL with ASP, MSSQL with ASP, and PostgreSQL with PHP.
Tags: addresses, almost, appears, changing, Data, Databases, dynamically, efficient, example, help, hosting, hundreds, information, product, visitors, Web, web pages
Posted in web hosting | No Comments »