RVData questions

[Cosmin33]

Hi,

1. Is it RVData thread safe when 2 or more separate threads are simultaneously accessing it (most of the time reading from items)?
2. the function which copy RVData.Items (if it's Unicode text) into a widestring variable can it be modified to write the data directly into a memory area (Byte array, WideChar array, WideString of a specified length and so on, at a specific position)?
In the current implementation time and memory is wasted by having to write the data into that widestring variable then into the memory area. When you make a lot of "calls" like my app does (hundreds of millions), it definitely matter to improve the time and to prevent memory fragmentation.

Thank you.

Kind regards,
Cosmin

[Sergey Tkachenko]

1. If you just read data, without modifying them you can do it from multiple threads.

2. When I tested speed of TRVUnicodeString (which is WideString in Delphi 5-2007 and UnicodeString in newer version of Delphi), I tested multiple operations like S := S + 'new string', I found that
- WideString is very slow, much slower than writing to Stream
- UnicodeString is very fast.

There are the following functions for getting data as a string:
1. Functions from RVGetTextW.pas, like GetAllText or GetRVDataText. They are slow in Delphi 5-2007 and fast in new versions of Delphi.
2. Functions from RVLinear.pas, RVGetTextLength and RVGetTextRange. They are fast in all versions of Delphi, because for older version of Delphi they use a memory stream.
3. Method SaveTextToStreamW: fast. I recommend saving to TRVMemoryStream (defined in RVClasses.pas)

[Cosmin33]

Thank you.

1. Yes and no. I intend to move the code for getting the text and the search into text code into a few separate threads (depending on number of processors) but the code to write the changes into the srv/rve will still be in the main thread.
So I will have to restrict the threads when the main thread is writing.
2. You said widestring is very slow on D7. This is true also with array of widechar?
I noticed array of Byte is very fast in many Delphis. Also Pointer to array of Byte.
How about writing the widestring in an array of Byte ? But I will still need a fast way of combining 2 Bytes in a WideChar and splitting a WideChar into 2 bytes.

Thank you for the memory stream tip.
Maybe it will help me speed up other code too.

Kind regards,
Cosmin

[Sergey Tkachenko]

1.
Changes in content occur:
- between calls Clear and Format (threads must pause their work before Clear and resume after Format). Usually you call these methods yourself so this is not a problem.
- from the call of TCustomRichViewEdit.BeforeChange (but only if returns True) till the call of TCustomRichViewEdit.DoChange. You can use OnChanging .. OnChange events to pause/resume your threads.

2. When you call s := s + 'new string', memory for old value of s is freed, memory for new value of s is allocated.
This operation is very slow for D7's WideString, and very fast for new UnicodeString, probably because UnicodeString is managed by new advanced Delphi memory manager.

You do not want to free and reallocate memory many times while saving TRichView to a string?
Ok, we need to allocate some buffer, write to it, and increase the buffer size (reallocate it) only on overflow.
But this is how TMemoryStream works! You do not need to reinvent the wheel with array of bytes, everything is already implemented in a convenient way in TMemoryStream (or, better, TRVMemoryStream from RVClasses.pas - it allocates larger buffers, so it is faster than TMemoryStream).
See my previous suggestions.

[Cosmin33]

You do not want to free and reallocate memory many times while saving TRichView to a string?
Ok, we need to allocate some buffer, write to it, and increase the buffer size (reallocate it) only on overflow.

Does RVData stores also the length of the text items? Because you can calculate very fast the length of the buffer and set it before starting to copy in it.
This way everybody is happy.

By the way, I use s := s + 'new string' only on very small strings.
On large strings I use SetLength + Move or SetString(.., copy_from, ...). They are much more effective.
Also I have found a project on Github that was trying ddabove to replace functions like Move with faster ones made with calls to MMX/SSE.
In some situations there was a big speedup.
FastCode was its name. It was way ahead of its times.
But:
1. Over the years I noticed some of the functions aren't perfect.
2. The speedups were on Intel processors, while on AMD was very small, or in some cases, SMALLER than the original speed.

[Sergey Tkachenko]

Calculating a buffer size is not so trivial, considering that you need to process non-text items.
And different text saving functions do it differently:
- SaveTextToStreamW method and the functions from RVGetTextW.pas calls methods of non-text items for text saving, so you cannot know the text size before saving it.
- The functions from RVLinear.pas store all non-text items (except for tables) as 1 character.

If you use functions from RVLinear pas, there is a function that allows calculating the necessary buffer size: RVGetTextLength. But still, the function for getting text (RVGetTextRange) cannot write to the buffer (as I said before, it uses TRVMemoryStream for Delphi 5-2007, and s := s + newstring in newer versions of Delphi).

As I said, I did a measurement and found that for UnicodeString, s := s + newstring is faster than streams or TStringBuilder (because of use of FastMM memory manager + no overhead for calling functions)

[Cosmin33]

Actually I process only tabs from nontext items.
So, if I would have access to stored size/length of text items, it would be easy to calculate final size.

If I would use an TRVMemoryStream and SaveTextToStreamW, the text would be copied as a AnsiChar/Byte array in the stream. So I would still need a way to read/write array elements as WideChars.
This could work, although I am not sure about the speed:
var
P: PWideChar;
MS: TRVMemoryStrream;
begin
MS := TRVMemoryStream.Create;
MS.Size := 1048576;
MS.Position := 0;
//here copy text and tab items from paragraph

//Now accessing the first element from WideChar array, not sure if I have to add/sub 1 to MS.Memory
P := PWideChar(@MS.Memory);
//P[0] first, P[1] second and so on...