mksqlite  2.5
A MATLAB interface to SQLite
MD5 hashing and packaging
sqlite_test_md5_and_packaging

Contents

function sqlite_test_md5_and_packaging

    clear all
    close all
    clc
    dummy = mksqlite('version mex');
    fprintf( '\n\n' );


    mksqlite( 'open', '' );

mksqlite Version 2.5 build: 133, ein MATLAB Interface zu SQLite
(c) 2008-2017 by Martin Kortmann <mail@kortmann.de>
                 Andreas Martin  <andimartin@users.sourceforge.net>
basierend auf SQLite Version 3.16.2 - http://www.sqlite.org
mksqlite verwendet:
 - DEELX perl kompatible regex engine Version 1.3 (Sswater@gmail.com)
 - BLOSC/LZ4 1.3.0-rc3.dev zur Datenkompression (Francesc Alted / Yann Collett) 
 - MD5 Message-Digest Algorithm (RFC 1321) Implementierung von Alexander Peslyak
   
Platform: PCWIN64, little endian

MD5 Test Suite

(ref: http://tools.ietf.org/html/rfc1321)

    % Compare md5 hashing values with the reference values, listed from the above source
    clc
    fprintf( 'MD5 Test Suite:\n\n' );
    q = mksqlite( 'select MD5(?) as MD5', 'a' );
    fprintf( '"a":\n%s\n%s\n\n', lower( q.MD5 ), '0cc175b9c0f1b6a831c399e269772661' );
    q = mksqlite( 'select MD5(?) as MD5', 'abc' );
    fprintf( '"abc":\n%s\n%s\n\n', lower( q.MD5 ), '900150983cd24fb0d6963f7d28e17f72' );
    q = mksqlite( 'select MD5(?) as MD5', 'message digest' );
    fprintf( '"message digest":\n%s\n%s\n\n', lower( q.MD5 ), 'f96b697d7cb7938d525a2f31aaf161d0' );
    q = mksqlite( 'select MD5(?) as MD5', 'abcdefghijklmnopqrstuvwxyz' );
    fprintf( '"abcdefghijklmnopqrstuvwxyz":\n%s\n%s\n\n', lower( q.MD5 ), 'c3fcd3d76192e4007dfb496cca67e13b' );
    q = mksqlite( 'select MD5(?) as MD5', 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789' );
    fprintf( '"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789":\n%s\n%s\n\n', lower( q.MD5 ), 'd174ab98d277d9f5a5611c2c9f419d9f' );
    q = mksqlite( 'select MD5(?) as MD5', '12345678901234567890123456789012345678901234567890123456789012345678901234567890' );
    fprintf( '"12345678901234567890123456789012345678901234567890123456789012345678901234567890":\n%s\n%s\n\n', lower( q.MD5 ), '57edf4a22be3c955ac49da2e2107b67a' );

MD5 Test Suite:

"a":
0cc175b9c0f1b6a831c399e269772661
0cc175b9c0f1b6a831c399e269772661

"abc":
900150983cd24fb0d6963f7d28e17f72
900150983cd24fb0d6963f7d28e17f72

"message digest":
f96b697d7cb7938d525a2f31aaf161d0
f96b697d7cb7938d525a2f31aaf161d0

"abcdefghijklmnopqrstuvwxyz":
c3fcd3d76192e4007dfb496cca67e13b
c3fcd3d76192e4007dfb496cca67e13b

"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789":
d174ab98d277d9f5a5611c2c9f419d9f
d174ab98d277d9f5a5611c2c9f419d9f

"12345678901234567890123456789012345678901234567890123456789012345678901234567890":
57edf4a22be3c955ac49da2e2107b67a
57edf4a22be3c955ac49da2e2107b67a

Packaging time for random numbers

    fprintf( 'BLOSC test suite:\n\n' );

    compressor = 'blosclz';   % blosclz, lz4, lz4hc, qlin16, qlog16

    % Using typed BLOBs with compression feature
    mksqlite( 'typedBLOBs', 2 );
    mksqlite( 'compression_check', 1 );

BLOSC test suite:

1.000.000 random numbers

    data = randn( 1e6, 1 );

    for level = 9:-1:0
        % set compressor and compression level
        mksqlite( 'compression', compressor, level );

        % measuring
        q = mksqlite( ['SELECT BDCPackTime(?) AS t_pack, ', ...
                       '       BDCUnpackTime(?) AS t_unpack, ', ...
                       '       BDCRatio(?) AS ratio, ', ...
                       '       ? as data'], ...
                       data, data, data, data );

        % display results
        fprintf( 'Level %d: pack(%gs), unpack(%gs), ratio(%g%%)\n', ...
                  level, q.t_pack, q.t_unpack, q.ratio*100 );
    end

Level 9: pack(0.0443139s), unpack(0.00764704s), ratio(96.6089%)
Level 8: pack(0.0412052s), unpack(0.00774793s), ratio(96.5709%)
Level 7: pack(0.0579164s), unpack(0.0173159s), ratio(96.5709%)
Level 6: pack(0.0363949s), unpack(0.00760716s), ratio(96.6098%)
Level 5: pack(0s), unpack(0s), ratio(100%)
Level 4: pack(0s), unpack(0s), ratio(100%)
Level 3: pack(0s), unpack(0s), ratio(100%)
Level 2: pack(0s), unpack(0s), ratio(100%)
Level 1: pack(0s), unpack(0s), ratio(100%)
Level 0: pack(0s), unpack(0s), ratio(100%)

Same random numbers, cumulated

    data = cumsum( data );

    for level = 9:-1:0
        % set compressor and compression level
        mksqlite( 'compression', compressor, level );

        % measuring
        q = mksqlite( ['SELECT BDCPackTime(?) AS t_pack, ', ...
                       '       BDCUnpackTime(?) AS t_unpack, ', ...
                       '       BDCRatio(?) AS ratio, ', ...
                       '       ? as data'], ...
                       data, data, data, data );

        % display results
        fprintf( 'Level %d: pack(%gs), unpack(%gs), ratio(%g%%)\n', ...
                  level, q.t_pack, q.t_unpack, q.ratio*100 );
    end

Level 9: pack(0.0342115s), unpack(0.00507916s), ratio(78.3423%)
Level 8: pack(0.0315091s), unpack(0.00496361s), ratio(78.2828%)
Level 7: pack(0.0292506s), unpack(0.00496185s), ratio(78.2387%)
Level 6: pack(0.0274068s), unpack(0.00505423s), ratio(78.2683%)
Level 5: pack(0.0239368s), unpack(0.00471169s), ratio(78.5139%)
Level 4: pack(0.0234036s), unpack(0.00606338s), ratio(78.7939%)
Level 3: pack(0.0107308s), unpack(0.00342042s), ratio(83.2141%)
Level 2: pack(0.00918232s), unpack(0.00331778s), ratio(84.0885%)
Level 1: pack(0.00940081s), unpack(0.0054481s), ratio(87.4197%)
Level 0: pack(0s), unpack(0s), ratio(100%)

Same random numbers, half constant 0

    data( 1:500000 ) = 0;

    for level = 9:-1:0
        % set compressor and compression level
        mksqlite( 'compression', compressor, level );

        % measuring
        q = mksqlite( ['SELECT BDCPackTime(?) AS t_pack, ', ...
                       '       BDCUnpackTime(?) AS t_unpack, ', ...
                       '       BDCRatio(?) AS ratio, ', ...
                       '       ? as data'], ...
                       data, data, data, data );

        % display results
        fprintf( 'Level %d: pack(%gs), unpack(%gs), ratio(%g%%)\n', ...
                  level, q.t_pack, q.t_unpack, q.ratio*100 );
    end

Level 9: pack(0.0183787s), unpack(0.00375416s), ratio(39.2673%)
Level 8: pack(0.016925s), unpack(0.00371311s), ratio(39.2066%)
Level 7: pack(0.0156906s), unpack(0.0036917s), ratio(39.1877%)
Level 6: pack(0.0147081s), unpack(0.00357732s), ratio(39.2523%)
Level 5: pack(0.0129429s), unpack(0.00349022s), ratio(39.4758%)
Level 4: pack(0.0104094s), unpack(0.00341573s), ratio(39.5842%)
Level 3: pack(0.00642088s), unpack(0.00300662s), ratio(41.4027%)
Level 2: pack(0.00572846s), unpack(0.00296087s), ratio(41.8961%)
Level 1: pack(0.00586718s), unpack(0.00276525s), ratio(43.8939%)
Level 0: pack(0s), unpack(0s), ratio(100%)

QLIN16/QLOG16 test suite

    fprintf( '\nLossy compression "QLIN16":\n(Unique resolution over entire value range)\n' );

    level = 1; % Always 1, ratio is always constant 1:4
    data = cumsum( randn( 1e6, 1 ) );

Lossy compression "QLIN16":
(Unique resolution over entire value range)

Measuring QLIN16

    % set compressor and compression level
    mksqlite( 'compression', 'QLIN16', level );
    q = mksqlite( ['SELECT BDCPackTime(?) AS t_pack, ', ...
                   '       BDCUnpackTime(?) AS t_unpack, ', ...
                   '       BDCRatio(?) AS ratio, ', ...
                   '       ? as data'], ...
                   data, data, data, data );

Display results

    fprintf( 'Level %d: pack(%gs), unpack(%gs), ratio(%g%%)\n', ...
              level, q.t_pack, q.t_unpack, q.ratio*100 );

    figure
    plot( data, 'r-', 'linewidth', 2, 'displayname', 'Original' ), hold all
    plot( q.data, 'k-', 'displayname', 'Copy' )
    grid
    legend( 'show' )
    title( 'QLIN16' );

    fprintf( '\nLossy compression "QLOG16":\n(Lesser resolution for bigger values)\n' );

    level = 1; % Always 1, ratio is always constant 1:4
    data = data - min(data); % Only positive numbers allowed (and 0, NaN, +Inf und -Inf)

Level 1: pack(0.00981197s), unpack(0.00241245s), ratio(25.0001%)

Lossy compression "QLOG16":
(Lesser resolution for bigger values)

Measuring QLOG16

    % set compressor and compression level
    mksqlite( 'compression', 'QLOG16', level );
    q = mksqlite( ['SELECT BDCPackTime(?) AS t_pack, ', ...
                   '       BDCUnpackTime(?) AS t_unpack, ', ...
                   '       BDCRatio(?) AS ratio, ', ...
                   '       ? as data'], ...
                   data, data, data, data );

Display results

    fprintf( 'Level %d: pack(%gs), unpack(%gs), ratio(%g%%)\n', ...
              level, q.t_pack, q.t_unpack, q.ratio*100 );

    figure
    plot( data, 'r-', 'linewidth', 2, 'displayname', 'Original' ), hold all
    plot( q.data, 'k-', 'displayname', 'Copy' )
    grid
    legend( 'show' )
    title( 'QLOG16' );

Level 1: pack(0.0201017s), unpack(0.0146941s), ratio(25.0001%)

Packaging time for screenshot data

    % Screenshot (figure) as (RGB-matrix)
    fprintf( '\nScreenshot data:\n' );

    h = figure;
    set( h, 'units', 'normalized', 'position', [0.5,0.5,0.2,0.2] );
    x = linspace(0,2*pi,20);
    plot( x, sin(x), 'r-', 'linewidth', 2 );
    legend on
    F = getframe(h);
    delete(h);
    data = F.cdata;

Screenshot data:

Measuring

    level = 9;
    mksqlite( 'compression', compressor, level );

    q = mksqlite( ['SELECT BDCPackTime(?) AS t_pack, ', ...
                   '       BDCUnpackTime(?) AS t_unpack, ', ...
                   '       BDCRatio(?) AS ratio'], ...
                   data, data, data );

Display results

    fprintf( 'Level %d: pack(%gs), unpack(%gs), ratio(%g%%)\n', ...
              level, q.t_pack, q.t_unpack, q.ratio*100 );

Level 9: pack(0.000145169s), unpack(0.000116722s), ratio(3.30078%)

Close database

    mksqlite( 'close' );


Published with MATLAB® 7.13

Generated on Wed Apr 26 2017 21:41:45 for mksqlite by   doxygen 1.8.13