join vs format for giant strings? 
Author Message
 join vs format for giant strings?

Hi,

I any generating very long strings ( possibly in the megabyte size) and
I need to
wrap the string in a single pair of quotes.

I am currently doing

format "%s%s%s" "\'" $bigstring "\'"

I guess another approach might be

join [list "\'" %bigstring "\'" ] ""

Any idea on the fastest,  least memory intensive way to do the task?

Jerry



Wed, 28 May 2008 06:01:14 GMT  
 join vs format for giant strings?

Quote:

> Hi,

> I any generating very long strings ( possibly in the megabyte size) and
> I need to
> wrap the string in a single pair of quotes.

> I am currently doing

> format "%s%s%s" "\'" $bigstring "\'"

> I guess another approach might be

> join [list "\'" %bigstring "\'" ] ""

> Any idea on the fastest,  least memory intensive way to do the task?

Or just:

set bigstringwithquotes '$bigstring'

should work fine too (a single quote is not a special character to Tcl). You can
use [time] to look at the speed:

% for {set i 0} {$i < 1000000} {incr i} {
   append test x

Quote:
}

% time { string length '$test' } 100
5104.5 microseconds per iteration
% time { string length [format %s%s%s ' $test '] } 100
5124.72 microseconds per iteration
% time { string length [join [list ' $test '] ""] } 100
5027.5 microseconds per iteration

Looks to me like there is nothing much between them in terms of speed, so just
use whichever you prefer. I wouldn't expect the memory use to be much different
between them either (a megabyte isn't really very big these days).

Cheers,

Neil



Wed, 28 May 2008 06:16:27 GMT  
 join vs format for giant strings?

Quote:

> Hi,

> I any generating very long strings ( possibly in the megabyte size) and
> I need to
> wrap the string in a single pair of quotes.

> I am currently doing

> format "%s%s%s" "\'" $bigstring "\'"

> I guess another approach might be

> join [list "\'" %bigstring "\'" ] ""

> Any idea on the fastest,  least memory intensive way to do the task?

The join is the wrong thing to use -- or in this case abuse.

You could use the format, but you don't need the backslash in front of the
single quotes.  In other words you could just write it as:

     format "%s%s%s" "'" $bigstring "'"

But a better approach would be just write it as:
     "'$bigstring'"

However, I suspect you are going down a road with a pitfall in it.  Exactly
what are you attempting to do -- why do you want to wrap the string in a
pair of single quotes?



Wed, 28 May 2008 06:24:31 GMT  
 join vs format for giant strings?

Quote:

> Hi,

> I any generating very long strings ( possibly in the megabyte size) and
> I need to
> wrap the string in a single pair of quotes.

> I am currently doing

> format "%s%s%s" "\'" $bigstring "\'"

You don't need to escape the single quotes.

Quote:
> I guess another approach might be

> join [list "\'" %bigstring "\'" ] ""

I don't see the point in copying the string to a list just to convert
the list back to a string.

Another solution:

     set result "'$bigstring'"

I like that one the best because it's obvious what you're trying to
accomplish with a minimum of syntactic noise.



Wed, 28 May 2008 06:26:55 GMT  
 join vs format for giant strings?
Hi,

I need to wrap the string in single quotes in order to build a query
that
inserts the string into a database.

The string is an especially escaped contents of a binary file ( a tiff,
movie
or whatever).

Jerry



Wed, 28 May 2008 06:32:22 GMT  
 join vs format for giant strings?

Quote:
>set bigstringwithquotes '$bigstring'

Doh! Sometimes I wonder If it is time to hang it up!

Jerry



Wed, 28 May 2008 06:36:13 GMT  
 join vs format for giant strings?

[...]

Quote:
> I need to wrap the string in single quotes in order to build a query
> that inserts the string into a database.

> The string is an especially escaped contents of a binary file ( a
> tiff, movie or whatever).

Doesn't the database API do this for you?  (In the mysql and sqlite
ones, there's no need to use single quotes, anyway.)


Wed, 28 May 2008 08:04:47 GMT  
 join vs format for giant strings?

Quote:
>Doesn't the database API do this for you?  (In the mysql and sqlite
>ones, there's no need to use single quotes, anyway.)

There is a 'binary' interface but the tcl implementation cannot
handle NULL or DEFAULT. So I have to pass stuff as strings.

I am working with Postgresql and Pgtcl

Jerry



Wed, 28 May 2008 08:38:08 GMT  
 join vs format for giant strings?
Hi Jerry,

Recent versions of pgtcl support bound parameters.  Have a look at the
documentation for pg_exec if your version already implements this.
With bound parameters you don't need to convert your strings to SQL
literals at all.

benny



Thu, 29 May 2008 02:26:02 GMT  
 join vs format for giant strings?
Hi Jerry,

Recent versions of pgtcl support bound parameters.  Have a look at the
documentation for pg_exec if your version already implements this.
With bound parameters you don't need to convert your strings to SQL
literals at all.

benny



Thu, 29 May 2008 02:26:44 GMT  
 join vs format for giant strings?

Quote:

> Hi Jerry,

> Recent versions of pgtcl support bound parameters.  Have a look at the
> documentation for pg_exec if your version already implements this.
> With bound parameters you don't need to convert your strings to SQL
> literals at all.

> benny

yes, and even if you don't want to use bind vars, there is
[pg_escape_string]


Thu, 29 May 2008 04:27:29 GMT  
 join vs format for giant strings?

Quote:
>  Or just:

>  set bigstringwithquotes '$bigstring'

Depending on the usage of the result:
    set bigstringwithquotes '[string map {' \\'} $bigstring]'

--
Glenn Jackman
Ulterior Designer



Thu, 29 May 2008 13:11:46 GMT  
 join vs format for giant strings?
Hi Benny,

Unfortunately pgtcl support for bound variables does *not* support
NULL or DEFAULT so I have to go the string route.

Jerry



Thu, 29 May 2008 22:28:11 GMT  
 join vs format for giant strings?
Hi Jerry,

Quote:
Jerry writes:
> Unfortunately pgtcl support for bound variables does *not* support
> NULL or DEFAULT so I have to go the string route.

I rarely have that problem, if only because Tcl itself has the same
problem, that it can't represent NULL separately from the either "" or
"NULL".  So cases that need to use NULL will occur in separate code
branches anyway.

If I personally wanted to solve that problem, I would do this by using
a dynamic approach rather than try to encode a non-trivial string as
an SQL literal.

Something along the lines (untested):

  proc my_exec {db sql args} {
      set non_null_args {}
      set replace_list {}
      set idx_in_cmd 1
      set idx_in_args 1
      foreach arg $args {
          switch -- $arg {
              NULL - DEFAULT {
                  lappend replace_list \$$idx_in_cmd $arg
                  incr idx_in_cmd
              }
              default {
                  lappend replace_list \$$idx_in_cmd \$$idx_in_args
                  lappend non_null_args $arg
                  incr idx_in_cmd
                  incr idx_in_args
              }
          }
      }
      if {$idx_in_cmd != $idx_in_args} {
          set sql [string map $replace_list $sql]
      }
      eval pg_exec $db $sql $non_null_args
  }

This will still not work with stuff where you need "field IS NULL"
instead of "field = 'literal'" but than I doubt that many other
implementations of bound parameters have that ;-).

benny



Fri, 30 May 2008 21:56:04 GMT  
 join vs format for giant strings?

Quote:

> Hi Benny,

> Unfortunately pgtcl support for bound variables does *not* support
> NULL or DEFAULT so I have to go the string route.

> Jerry

Unfortunately, alot depends on the underlining libpq lib. It looks like
support for NULL could be added to pgtcl, but not DEFAULT.

    --brett



Sat, 31 May 2008 00:46:43 GMT  
 
 [ 17 post ]  Go to page: [1] [2]

 Relevant Pages 

1. string.join() vs % and + operators

2. Oops: difference in operation of string.join and ''.join

3. Format Strings -- Real vs. Expected Behaviour

4. format specifiers in format string param.

5. GAS vs M$ vs Borland; formats questions

6. Stylistic question: returning strings vs. pointers to strings

7. string object methods vs string module functions

8. string.join question

9. join vs instances

10. string.join() syntax quirky?

11. string.join is abysmally slow

12. bug in string.join()?

 

 
Powered by phpBB® Forum Software