DeutschEnglish

Submenu

 - - - By CrazyStat - - -

7. October 2012

SQLite: escaping table and column names correctly

Filed under: DBMS,PHP,phpLiteAdmin — Tags: , , , , , , , , , , — Christopher Kramer @ 21:44

Summary:

Use double quotes around column names, table names, trigger names etc.

SELECT "column name", "my column", FROM "table name"

Use single quotes around string values

SELECT * FROM "table name" WHERE "column name" = 'string value'

Use two double quotes if e.g. your column name contains a double quote (e.g. my”column)

SELECT "my""column" FROM "my""table"

Details:

I joined the development team of phpliteadmin some months ago and am currently fixing several issues. One problem is caused by SQLite’s flexibility of column- and table names. A column named “SELECT”, “my column”, “this, that”, or even “this contains ‘single quotes'” is completely legal in SQLite. And what is legal is being used. So phpliteadmin needs to be able to handle these types of names as well, which is not really the case at the moment.

Let’s assume you query a column called “select”, you are likely to get errors like that in PHP:

Warning: SQLiteDatabase::query() [sqlitedatabase.query]: near “select”: syntax error in …

Or using PDO:

HY000 / 1 / near “select”: syntax error

From MySQL I knew backticks around table or column identifiers like in this example:

SELECT `select`, `my column`, `this, that` FROM `group by table`

(By the way, you see my syntax highlighter has problems with reserved words as column names as well ;-))

I tried this in SQLite and it worked, so I went that way. But there are different versions of SQLite and for the old version that, in PHP, is implemented by SQLiteDatabase, would give the following error:

Warning: SQLiteDatabase::query() [sqlitedatabase.query]: unrecognized token: “`” in …

So I found out that instead of backticks, normal single quotes work:

SELECT * FROM 'group by table'

This works well, but there is a problem here with column names:

SELECT 'select' FROM 'group by table'

This will not work as expected. It will not return the values of the column ‘select’, but create a column filled with the text ‘select’. That is of course standard SQL behaviour and shows that it is a bad idea to use single quotes around column or table names. The bug report by maitredede on the phpliteadmin issue tracker made me aware of another way to escape column and table names:

SELECT [select], [my column], [this, that] FROM [group by table]

This works very well, both with SQLiteDatabase, SQLite3 and PDO when accessing SQLite from PHP. I have not tried it using the SQLite C library directly, but it should work as well.

So I’d say this is the best way to do this as it works the same way with all versions of SQLite / all PHP SQLite extensions.

 

Update:

Well, probably I should have read the manual first:

SQlite supports ` for compatibility with MySQL and [] for compatibility with MS Access and SQL Server. But it also supports double quotes, which is standard SQL. So I’d say this is the best way to do it:

SELECT "select", "my column", "this, that" FROM "group by table"

It works with all SQLite versions / extensions just like the [] syntax. (And my syntax highlighter supports it as well ;-).)

Update 2:

Another thing that might be of interest: What to do, if a column/table name contains double quotes? Well, in this case duplicate them and you are done. Let’s assume you have a column called like this:

my “column

Then you query it like this:

SELECT "my""column" FROM "my_table"

Recommendation

Try my Open Source PHP visitor analytics script CrazyStat.

4. October 2012

PDO / sqlite: database table is locked

Filed under: DBMS,PHP,phpLiteAdmin — Tags: , , , , , , , — Christopher Kramer @ 22:06

At the moment I am working again on phpliteadmin, aย  php-based web GUI for database administration of sqlite databases. While debugging, I stumbled across a problem that only occurred with the PDO extension (not with SQLiteDatabase or SQLite3). I got the following error message while trying to drop a table:

HY000 / 6 / database table is locked

By the way, you can use PDO::errorInfo() to get these error messages. So as the error correctly explains, the table I tried to drop seemed to be in use. PDO documentation for PDO::query() also explains the problem (even though DROP TABLE statements are fired using PDO::exec()):

If you do not fetch all of the data in a result set before issuing your next call to PDO::query(), your call may fail. Call PDOStatement::closeCursor() to release the database resources associated with the PDOStatement object before issuing your next call to PDO::query().

So I hunted for the open cursor on a resultset of the table in question and could not find any. Finally, I found the SQL statement which still had an open cursor:

SELECT * FROM sqlite_master

Thinking about this, it is obvious: When querying sqlite_master, you request information about database tables. If one of the table gets altered or dropped, this might change the data listed in the resultset on which I still have a cursor.

Maybe this is a special case as usually you do not query sqlite_master a lot. But in case you do, this might be useful information.

To solve the problem, as the manual says, simply release the cursor using PDOStatement::closeCursor() before dropping/altering tables.

30. June 2012

CrazyStat counting lots of hits / visits

Filed under: CrazyStat,PHP — Tags: , , , , , — Christopher Kramer @ 16:52

Lately, I got a screenshot of CrazyStat counting quite a lot of visitors and hits. With this post, I’d like to share it with you. It’s quite impressive to see that CrazyStat is able to cope with that many hits easily.

With 4 000 hits per day on average within the last month and about 17 000 visitors on 26th of June, I consider this website quite successful!

So how is your site doing? I’d love to see other screenshots of CrazyStat counting lots of hits.

 

CrazyStat with lot of visitors

Screenshot of CrazyStat counting lots of visitors

9. June 2012

Typo3 and other charsets than UTF-8 (latin1 / ISO-8859-1, …)

Filed under: PHP,Server Administration,Typo3 — Tags: , , , , , , , — Christopher Kramer @ 12:30

When updating a Typo3 installation to Typo3 4.5.x, I had problems with charsets and explained the solution here.

Now updating an installation of Typo3 to 4.6.x, I ran into another charset problem: The backend now was completely UTF-8 and therefore, changing texts in the backend caused them to be stored as UTF-8. As the frontend was still ISO-8859-1, special characters (Umlaute) over there got messed up. Maybe there is a way out of this as well ($TYPO3_CONF_VARS['BE']['forceCharset'] I guess), but this clearly shows that Typo3-developers drop support for other charsets slowly and that it might be easier to switch to UTF-8.

In the release notes of Typo3 4.5, I found the following passage:

UTF8 by default: New installations will use UTF8 automatically. Keep in mind that we will be deprecating all other charsets in the release of 4.5, but still support those charsets. 4.7 or maybe even 4.6 will be the first “UTF-8 only” release. When upgrading from older releases to 4.5, you will have to specifically set $TYPO3_CONF_VARS['BE']['forceCharset'] and $TYPO3_CONF_VARS['BE']['setDBinit'] in your localconf.php. An Upgrade Wizard will help you with that.

In the release notes of Typo3 4.6, I could not find a word about UTF-8, but in the release notes of 4.7, it is clearly stated:

check you database if it is utf-8 encoded – TYPO3 4.7 only will work with utf-8.
[…]
The forceCharset option has been deprecated in version 4.5. UTF-8 is now enforced. Even though other values than “utf-8” have not been possible anymore for some time, the option’s value has been queried at plenty of places within the whole core. These references, the option in the Install Tool, as well as many defaults with charset โ€œiso-8859-1โ€ in several classes have been changed, so TYPO3 now works UTF-8-only internally.

So it is clearly time to make the switch.

It is not that complicated – everything is described very well over here.

As the official wiki is very long and explains lots of stuff you might just not care, here are the basic steps:

  • Backup Database and Files
  • Set the charset in your webserver (e.g. “AddDefaultCharset utf-8” in a .htaccess)
  • Adjust some settings in localconf.php:
    // For backend charset
     $TYPO3_CONF_VARS['BE']['forceCharset'] = 'utf-8';
     $TYPO3_CONF_VARS['SYS']['setDBinit'] = 'SET NAMES utf8;'; 
    
     // For GIFBUILDER support
     // Set it to 'iconv' or 'mbstring'
     $TYPO3_CONF_VARS['SYS']['t3lib_cs_convMethod'] = 'mbstring';
     // For 'iconv' support you need at least PHP 5.
     $TYPO3_CONF_VARS['SYS']['t3lib_cs_utils'] = 'mbstring';
  • Adjust your typoScript (change language to your needs):
    config.locale_all = de_DE.utf-8
  • Convert your templatefiles to UTF-8 (and remap them if you use TemplaVoila) – usually in fileadmin/templates
  • Convert your DB to UTF-8
    1. Backup it first if you have not yet (believe me!)
    2. Paste this tool into fileadmin
    3. Run it by opening it in the browser (http://example.com/fileadmin/db_utf8_fix.php)
    4. If everything says “OK”, change the constant “SIMULATE” to false
    5. Run it again
    6. Clean cache of Typo3
    7. Check your site (esp. special characters). If the content is messed up or parts are missing, do the following:
      1. Restore the backup of the database (yes, I told you!)
      2. Uncomment lines 108 – 123 in db_utf8_fix.php
      3. Run it in browser againClean cache in Typo3
    8. Clean all cache in Typo3 Backend

You can find more detailed information here. There are also lots of other ways described how to convert the database.

Happy converting!

 

Update 2014-05-05: Changed link to db_utf8_fix-script as the original site is reported to be attacked and does not host the script anymore. I cannot check if the script at snipplr is exactly the same, but it looks so.

30. May 2012

Released: phpLiteAdmin 1.9.2 includes CSV import/export

Filed under: DBMS,PHP,phpLiteAdmin,Server Administration — Tags: , , , , , , — Christopher Kramer @ 15:54

As I wrote in March, I implemented CSV import and fixed export issues of phpLiteAdmin. This fixed phpLiteAdmin bug #71. I also wrote a small fix for bug #75. Today, new version 1.9.2 of phpLiteAdmin was released including both fixes. You can download it here.

I’d like to thank the phpLiteAdmin team for including my work and allowing me to join the team. I plan to address more issues of phpLiteAdmin in the future to push phpLiteAdmin a little further. There is still some more work to be done which I will have a look at once I find the time.

I recommend the new version to anybody using phpLiteAdmin (and also everybody who doesn’t yet ;-)). Please use the bugtracker in case you find any issues.

I hope some of you find the new features useful or are happy to see those bugs fixed.

Thanks again to the phpLiteAdmin team for the great tool and the opportunity to contribute to the project. Fortunately, I do not have to create a fork to improve the tool.

 

28. April 2012

CrazyStat 1.71 RC1 released

Filed under: CrazyStat,PHP — Tags: , , , , , , , — Christopher Kramer @ 20:31

Today I released CrazyStat 1.71 RC1, the first Release Candidate of version 1.71. As there have already been two beta-versions tested by some testers, I consider this release candidate quite stable. I expect that no or only small changes will be made for the final version. Therefore, I’d like to encourage all CrazyStat users to update CrazyStat to 1.71 RC1. It is very simple, as described in the FAQ.

So what are the major changes?

  • New languages: Russian and Danish. Thanks a lot to the translators Vladimir and Liza! I hope CrazyStat will be translated into more languages soon. It is really not much work. Please contact me if you are interested.
  • PHP4 support dropped. I removed all workarounds for old PHP versions. Now at least PHP 5.1 is required. If your host is still running PHP4, you should really update or change to another hoster. PHP4 is not maintained for quite a while now. Also support for old GDlib-versions was dropped (1.x).
  • You can now link the entries of the file-module to the files. See the config setting you need to set.
  • IP anonymization reworked. This is now turned on by default. See FAQ for more information.
  • Average and total visiting time in hits-module
  • New logo by kartoffelpfluecker (thanks!)
  • Improved UTF8-support
  • New Browsers (Firefox-Versions), Bots, Operating Systems etc.
  • Lots of (smaller) fixes, i.e. producing PHP notice errors or invalid (X)HTML
  • Fix of a bug that caused weird ordering of the days-module sometimes
  • Fix of a bug that made Chrome transfer passwords unencrypted and corrupt the password file when changing passwords using Chrome

So this is more a maintenance release, not a major whole new version. See history for more details (with CrazyStat’s new public SVN hosted on Sourceforge, you can even track every individual change).

I hope you enjoy the new version. You can download it here.

Tell me what you think about it in the comments. In case you have problems, please ask in the support forum.

Enjoy using CrazyStat!
Christopher

26. April 2012

How to include CrazyStat in WordPress

Filed under: CrazyStat,PHP,Wordpress — Tags: , , , , , — Christopher Kramer @ 00:52

Of course it is possible to include CrazyStat in wordpress. I will describe here how it can be done. This approach assumes you have your own wordpress installation and can edit templates. It might not work this way if your blog is hosted on wordpress.com or similar. But I guess using CrazyStat in these cases would still be possible. Contact me if you need to know how and I will have a closer look.

So I assume you have a running wordpress installation and installed CrazyStat like described in the readme (at least steps 1-6 of quick install instructions). I assume you have a backup of your wordpress installation (at least the themes folder).

So the question is where to put the include code:

<?php @include_once("stat/src/include.php"); ?>

Log into the wordpress admin panel and click appearance/editor (German: Design/Editor). Make sure you selected the correct theme in the dropdown menu on the top right.

In case you do not want to use a counter

Click on “footer.php” on the right. You will see some HTML / PHP code in the editor. Put your cursor just before </body> (which should be quite at the end).ย  This is the best place to insert the CrazyStat code:

<?php @include_once("stat/src/include.php"); ?>

The path is seen from the wordpress root-directory. This code assumes you installed CrazyStat in the wordpress root-directory under “stat”. In case you installed it one level higher, you would need to use:

<?php @include_once("../stat/src/include.php"); ?>

Save your changes.

You might also need to adjust $config_rel_path. See config-documentation for details.

In case you want to use a counter

In this case, you’d need to find the place in your template where you want to place the counter. This should normally be somewhere after <body> in header.php, before </body> in footer.php or in sidebar.php. Unfortunately you need some HTML experience to find the correct place here. Use the same code as above.

In case you have problems

Please ask in the CrazyStat support forum in case you have problems or questions (no registration required at the moment!).

I hope I could clarify this for some people. Maybe I will do a wordpress plugin sometime. If you think this would be a great idea, tell me in the comments.

Alternative option: Use the WordPress plugin mzz-include-CrazyStat (Update 2017-02-17)

The problems with the solution described above are:

  • Whenever the theme is updated, the change of the footer.php might get lost.
  • If you switch to another theme, you need to add the CrazyStat code again.

To solve these problems, CrazyStat user Morgan Jassen developed a small helper plugin that just inserts the include code:

Find it here: https://github.com/mjassen/mzz-include-crazystat

I just found this today and have not tried it yet, but think it is the better option. Thanks, Morgan!

17. April 2012

CrazyStat 1.70 detects Firefox 11 as 1.x (fix)

Filed under: CrazyStat,PHP — Tags: , , , , — Christopher Kramer @ 00:25

Hey,

CrazyStat version 1.70 (and before) detects Firefox version 11 (and upwards) as “Firefox 1.x”.

Firefox versions were really coming up fast lately, so I did not finish version 1.71 to get all those new versions added before they come out.

To fix this issue in 1.70 (or before), simply add “Firefox/11=Firefox 11” to stat/usr/keywords/browser.txt or even better download the current browser.txt from SVN. Just replace the file and clean the cache.

Note that you can easily customize browser detection as described in FAQ.

Of course this won’t be an issue in upcoming 1.71.

Greetings,
Christopher

21. March 2012

Public SVN for CrazyStat

Filed under: CrazyStat — Tags: , , , — Christopher Kramer @ 23:12

Finally, CrazyStat’s SVN has gone public!

CrazyStat uses Sourceforge for SVN hosting. You can browse it here:

https://sourceforge.net/p/crazystat/code/

To checkout the current development version of CrazyStat, use one of these:

  • SVN over HTTP protocol:
    svn checkout http://svn.code.sf.net/p/crazystat/code/trunk crazystat-code
  • SVN protocol:
    svn checkout svn://svn.code.sf.net/p/crazystat/code/trunk crazystat-code

In the SVN repository, you can find the current CrazyStat 1.71 development version (in the trunk) along with 1.71 beta1 and beta2.

You can also read the commit logs to see what is going on with CrazyStat development.

In case you want to test the upcoming CrazyStat version 1.71, I’d recommend checking out the current development version from the trunk. Please let me know if you find any bugs or issues with the current development version (which will soon become RC1…).

This is part of my plans to make development of CrazyStat more open.

Greetings!

10. March 2012

phpliteadmin: CSV import and export

Filed under: DBMS,PHP,phpLiteAdmin,Server Administration — Tags: , , , , , , , , — Christopher Kramer @ 12:24

In case you use sqlite, a serverless, transactional SQL-complete database engine, you might also know phpliteadmin. It is a web-based DB admin tool just like phpMyAdmin is for MySQL. It is small and does not require complex installation just like sqlite. It is great to do most simple operations on a sqlite database, but in my opinion it still has some important limitations and bugs.

One thing I recently missed was the possibility to import CSV files into a SQLite database. The SQLite shell makes this very easy (if the file has the correct syntax). But this has also some limitations and you have to leave the phpLiteAdmin GUI and start a SQLite shell. So I thought it should be part of phpLiteAdmin. It already had some GUI options for CSV import, but the actual import implementation was missing, so I implemented CSV import myself. On the way testing the exported csv and importing it again I noticed some export bugs of phpliteadmin like this one and fixed them on the way.

So those are the changes I did to phpLiteAdmin:

Import:

  • implemented CSV import with some features like:
  • you can define Enclosure, Escaper, Null and first-row-field-description just like when exporting
  • using PHP’s fgetcsv()
  • can import files that were exported with phpliteadmin using the same settings!
  • can even import files exported with phpliteadmin without my corrections and improvements (see below), although they are not “standard CSV”.

Export:

  • removed line terminator option because:
    • this was not implemented
    • in my opinion anything else than a linebreak does not make sense, so useless feature
  • Fix: Tables with indexes were exported multiple times
  • Fix: Last column was terminated which is not usual in CSV
  • NULLs are not enclosed anymore to allow distinction between “NULL”-text
  • FIX: Removing CRLF was not implemented
  • FIX: Tables that were not exported caused additional linebreaks in output

You can find anything related to these changes in phpLiteAdmin’s bug tracker.

You can also get my version with the improvements from the bugtracker or download it here.

I really hope this will make it into the next release of phpLiteAdmin, but unfortunately I did not get any feedback by the developers of phpLiteAdmin yet. Hopefully this is not getting a fork ๐Ÿ˜‰

I hope you like these features and bugfixes. You can give me feedback here or in the phpliteadmin bug tracker.

I really think a good database management tool for SQLite is needed and phpLiteAdmin has the potential to become one. We will see.

You can also tell me if you use some other management tool for SQLite which you think is way better ๐Ÿ˜‰

Update 2012/05/13: The phpliteadmin developers just announced that an official new version including these improvements will come up soon. Great news, I am looking forward to this new version!

« Newer PostsOlder Posts »