Messages posted by: saoj

Nope. Bugs can be reported and described here. There are currently no outstanding bugs I am aware of.

amitfrid wrote:Hi,
Can you make a tag of the change in SVN? some way that I can identify what is the version of the code I am using.

Thanks,
Amit

The SVN Revision containing that change is #101

amitfrid wrote:Hi,
I can't see the fix in SimpleLogger there.

Thanks,
Amit

You must me hitting another jar or some kind of cache. The change is here: http://www.space4j.org/beta/space4j.jar. The beta jar is stable. This is the only change so far.

It looks like someone forked a space4j project and did some modifications: http://nicolas.delsaux.free.fr/space4j/apidocs/index.html?org/space4j/Space4J.html

It is totally fine to do that, besides the fact that it confuses people about the official space4j version.

Not sure where you saw version 0.9.7. The current version is 0.9.2. This feature was never done before. So I have committed the modification to allow that, and you can download the beta jar here: http://www.space4j.org/beta/space4j.jar

You can pass the maxLogSizeBeforeRotation in the constructor or you can change the default through a static method. See below:

The forum problem was fixed. Answering your question in a few minutes...

Amit wrote:
Hi,
I am evaluating Space4J for my application.
I found one configuration which make my usage not scalable: log size is set to 10k in SimpleLogger, with no public API to modify it.
Creating lots of objects creates a very long list of log files in the file system. Although these object are for short-period and I am planning to write a cleaning utility for "pre-last-snaphot" log files, I would like to have the ability to create bigger log files (even for the sack of disk fragmentation).
I saw a newer 0.9.7 javadocs, with this ability, but the code is not in SVN (am I right?)

Thanks,
Amit.

P.S sorry for sending it as a private message, I couldn't post new topic in the forum, get sql exception of duplicate entries all the time...

No they are not, unless you want to replicate the logs in a database, in other words, unless when you are using the executeSQL method of your commands. Most of the times you can just delete the old logs before the last snapshot. A cleanup utility will be very handy.

dlouwers wrote:
That is a possibility but highly granular locking without some form of higher order abstraction is extremely error prone and almost sure to cause deadlock issues or other issues that arise from data modification not being atomic. What are your thoughts on letting S4J be a possible solution to the problem and offer a more strict isolation mode by optionally offering access through a ReentrantReadWriteLock? Sure it would be less performant than allowing concurrent read/writes and leave the nitty-gritty to the users. But it would be a nice option. This together with the advise to never let mutable objects leave the prevalent system could protect users from most if not all concurrency pitfalls.
And yes I know this takes time and effort. I am just mentioning this for the sake of discussion to see if you see the merit in it.

You convinced me about the merit of this.

Databases provide isolation levels, so S4J should provide too. LET'S DO IT. Actually, do you want to start coding something? I can provide you with commit access to svn. Just let me know your account on sourceforge.

dlouwers wrote:
Even though a database will allow concurrent reads with a single write in most default isolation levels, it will still make sure that writes to a field are atomic. A user will never read a half written date field for instance. Not so in code executed by the JVM. Take a long on a 32-bit JVM for instance. From what I understand writing to it is not atomic, but in fact consists of 2 processor operations. It is possible for another thread to read from this long when only it's lower or higher order bytes have been written. That is what I call, if not unexpected behavior, at least undesired behavior. To prevent stale cache on multiple cores I would mark these member variables as volatile anyway and that would do away with this danger but more complex mutable data would still need to have access synchronized. One way to deal with this using the tools at hand is treating all reads like a write, but that would eliminate the possibility to have concurrent reads. And that is the reason why I am wondering about the possibility of ReetrantReadWriteLock like behavior, because that addresses the issue very neatly without the need for more granular (and thus error prone) locking.

Like you said this is a classic JVM problem, not a S4J problem. The problem is: A mutable object shared by two or more read and write threads. Should the setters and getters be synchronized? Should the instance variables made volatile? When I tacked this problem some years ago the answer was: single-core = NO; multi-core = YES. Plus: long and double = YES (64-bit), the rest = NO (32-bit or less). I would expect by now that primitive reads inside the JVM be guaranteed to be atomic, but I am not sure.

So this is a Java / Architecture problem. If you think that might be an issue, synchronize properly the write and read operation on your mutable object, so it can be thread-safe not just inside the Space but anywhere it is used in a multithreaded environment.

I would never suggest pulling these object across the boundary! However the issue I am mentioning here exists inside the space as well.

When you fetch a object, you cannot make a clone/copy of the object. A relational database deal with values (columns) so it does not have this problem, but a object database deals with objects so I don't see how you can escape from that. The objects will be live and you can do whatever you want with them, even modify them, what would be disastrous. Perhaps we could give a little more thought in that area for Space4J. My first guess is that you want to be able to iterate through your lists and maps inside the Space. To have to create a copy/clone of every iterated object is going to be bad.

dlouwers wrote:
- From the FAQ I gather that reads are allowed concurrently with writes. Is this correct? I am asking this since reading the result of a half-finished write operation can cause unexpected behavior. If this is the case is it also possible to use a lock more akin to Java's ReentrantReadWriteLock to have all writes operate in isolation with possibility of a downgrade to a read lock?

Yes, reads are concurrent with writes, like any modern database. Now you have to define unexpected behavior. If you have a object User and you want to freeze it, one easy option is to clone it. The thing with S4J is when you fetch an object from the Space, that's a live object.

- Say a system would need 50GB of memory and has this readily available, wouldn't this much RAM totally smash performance of the garbage collector to the point of uselessness? Do you know of a solution to this bottleneck?

I wouldn't think so. Are you saying that it is too much memory for the GC to track? I think the modern GCs strategies would not be affected by that.

If you want to be sure, create a lot of memory, call System.gc(), log the GC details and play with the various types of GC available in the JVM. Or do some research about the GC algorithms. I don't know this from the top of my head.

cstamas wrote:

So, what is to be done with older snapshots and logfiles? Is it left to users as exercise to manage them?

You are right. This is left as an "exercise" for the user.

I mean: you can rotate them, archive them, delete them, etc.

This is pretty much system administration, right? I mean: does Apache take care of rotating their logs or this is something outside its domain?

Any contribution in that area for Space4J is appreciated.

skaak wrote:Thanks for the quick reply.

Does a cluster then improve performance?

Regards
skaak

Depends on how you see it.

It increases performance because it gives you LOAD BALANCE and ASYNCHRONOUS SNAPSHOTS.

But it also decreases performance due to replication of the commands to all nodes over the network.

But performance is always RELATIVE. You will still be very fast. As fast as or faster than related databases for example.

Check the stress test examples with cluster then you can draw your own conclusions.

skaak wrote:Hi, just a follow up on that. If I use a cluster, does it increase the RAM available to space4j?

No. A cluster gives you replication. It is like a RAID for storage. It is good for load balance, fault tolerance and asynchronous snapshots.

skaak wrote:
I haven't used it yet, but it would be nice if I can increase memory by adding nodes. I suppose that the size of the repository, if I can call it that, will rather be limited by the node with the smallest RAM?

The memory will be limited by the amount of RAM of the MASTER node. If the SLAVE nodes have more memory, that will not be used because modifications (WRITE) commands always happen first in the MASTER. If a slave have less memory, you may end up getting an OutOfMemoryException in the node, but that won't stop the whole system.

A best practice when using any cluster is to keep your system homogenous, in other words, to keep the machines in cluster as similar as possible.

If you need more memory you need more RAM, unless your JVM is not using all the available memory RAM from your machine. In that case you need to use the -Xms jvm options.

This patch was applied to HEAD on subversion.

Good catch!

-Sergio

pascal wrote:
Is there somewhere a list of current bugs ?

You can post bugs here.

We are trying to use multisort index and it works fine 10 min, then we observes that index are not updated correctly.

Make sure you are NOT updating your map outside commands. What you are describing does not look very likely to happen, because the index is updated inside the insert/delete/update commands, so it is either updated or an exception is thrown and the modification command is not executed.

We are doing concurrently a lot of insert/delete and update commands.

That's totally fine.

Are you sure you have really spotted a bug?