STL Tokenizer??? 
Author Message
 STL Tokenizer???

What is a good STL way to tokenize a string?  In the old stdlib there is
strtok() and Rouge Wave Tools.h++ has a split() function.  Is there any
equivalent functionality in the STL or Standard C++ Library?

Bill



Thu, 03 May 2001 03:00:00 GMT  
 STL Tokenizer???


Quote:
>What is a good STL way to tokenize a string?  In the old stdlib there is
>strtok() and Rouge Wave Tools.h++ has a split() function.  Is there any
>equivalent functionality in the STL or Standard C++ Library?

The closest thing is find_first_of and find_first_not_of, members of
the string class.

For example you could write something like this UNTESTED function:

std::list<std::string>
  GetTokens(std::string input, std::string delims)
{
  int t1;
  int t2 = 0;
  std::list<std::string> output;
  while ((t1 = input.find_first_not_of(delims, t2)) != input.npos)
  {
    t2 = input.find_first_of(delims, t1);
    if (t2 == input.npos)
      output.push_back(input.substr(t1, input.npos));
    else
      output.push_back(input.substr(t1, t2-t1));
  }
  return output;

Quote:
}

I think it would return the delimited strings, treating sequences of
adjacent delimiters as if they were one delimiter and skipping over a
delimiter sequence at the start.

--
Mike Enright

(Not speaking for Cetasoft)



Fri, 04 May 2001 03:00:00 GMT  
 
 [ 2 post ] 

 Relevant Pages 

1. Online references and question for 'tokenizer'

2. tokenizer/parsing

3. WANTED: C Tokenizer

4. string tokenizer

5. Help - String Tokenizer

6. HTML tokenizer (source code)

7. final note on tokenizers

8. std C++ string tokenizer class?

9. string tokenizer??

10. Does C# have a string tokenizer class?

11. C# String Tokenizer?

12. tokenizer or scanf() like functionality?

 

 
Powered by phpBB® Forum Software