Latest topics for the forum "General Questions"

Increasing the size of log files

saoj — GMT

The forum problem was fixed. Answering your question in a few minutes...

[quote=Amit]
Hi,
I am evaluating Space4J for my application.
I found one configuration which make my usage not scalable: log size is set to 10k in SimpleLogger, with no public API to modify it.
Creating lots of objects creates a very long list of log files in the file system. Although these object are for short-period and I am planning to write a cleaning utility for "pre-last-snaphot" log files, I would like to have the ability to create bigger log files (even for the sack of disk fragmentation).
I saw a newer 0.9.7 javadocs, with this ability, but the code is not in SVN (am I right?)

Thanks,
Amit.

P.S sorry for sending it as a private message, I couldn't post new topic in the forum, get sql exception of duplicate entries all the time...
[/quote]

Clarification : log and snapshots

yuwy — GMT

just a quick question about recovery. When a snapshot 006.snap is made, are the logs up to and including 006.log required anymore? I will be trying to write a little clean up process to limit the amount of old snaps/logs

Couple of questions regarding the system

dlouwers — GMT

Hi,

I have a couple of questions regarding the system and hope someone is willing and able to answer them:

- From the FAQ I gather that reads are allowed concurrently with writes. Is this correct? I am asking this since reading the result of a half-finished write operation can cause unexpected behavior. If this is the case is it also possible to use a lock more akin to Java's ReentrantReadWriteLock to have all writes operate in isolation with possibility of a downgrade to a read lock?
- Say a system would need 50GB of memory and has this readily available, wouldn't this much RAM totally smash performance of the garbage collector to the point of uselessness? Do you know of a solution to this bottleneck?

Thanks in advance,

Dirk Louwers

Space4j available thru Maven repository

cstamas — GMT

Hi there,

I read somewhere in this forum you are Ant-oriented ;) That's fine, but could you publish space4j artifacts (jar, sources and javadoc would be cool) to Maven Central repository, please?

It's de-facto standard, and is usable with Ant+Ivy too ... and would make a lot of developer lives easier.

I can recommend you this:
https://docs.sonatype.org/display/Repository/Sonatype+OSS+Maven+Repository+Usage+Guide

Thanks,
~t~

PS: and fix the site, since space4j.org is reported as nonexistent, but it should redirect to www.space4j.org to save some typing :)

Log trail and logfiles maintenance

cstamas — GMT

After reviewing the code, I did not find anything that "squashes" or removes the log files, something that lessens their count. As far as I understand, latest snapshot file + logs after it is all that is needed to load up the Space4j after boot, right?

So, what is to be done with older snapshots and logfiles? Is it left to users as exercise to manage them? ;)

Thanks,
~t~

KeyComparator and null values

niavloux — GMT

Hello,

I had some issues linked to indexing and I found out (I think) an issue in the KeyComparator regarding null values :
Let's consider we have a Key which contains only one value, which can be null.

Then, in the compare(Key key1, Key key2) method, if one of the parameters contains one value 'null', then the compareObjects(...) is called.
But this method always returns 0, so both keys are considered as equal.

This means that if key1 contains a 'null' value and key2 does not, the method returns 0, but that's wrong! Both keys are not equal.

I propose this correction :
[code]
protected int compareObjects(Object obj1, Object obj2)
{

if (obj1 == null && obj2 != null)
{
return -1;
}

if (obj1 != null && obj2 == null)
{
return 1;
}

return 0;
}
[/code]

Bug list

pascal — GMT

Hello,

Is there somewhere a list of current bugs ?
We are trying to use multisort index and it works fine 10 min, then we observes that index are not updated correctly.
We are doing concurrently a lot of insert/delete and update commands.
Has someone already encountered this kind of problem ?

Thank you for your answer.

Pascal.

Improvements on MULTI_SORTED index on 0.9.2

saoj — GMT

[color=red]Before:[/color]
[code]
@Override
public Collection findUsers(int minAge, int maxAge) {

// Remember that our index is of the type MULTI_SORTED.
// That's why we are doing this cast here.

MultiSortedMap map = (MultiSortedMap) indexByAge.getMap();

// A multi-sorted-map will store a Map for each key.
// That's why our index is NON-UNIQUE meaning the same
// key can be associated with many entries in the map.
// The map is also sorted by its key, meaning we can
// perform range searches efficiently.

ConcurrentNavigableMap subMap = map.subMap(new Key(minAge), true, new Key(maxAge), true);

Collection users = new ArrayList(subMap.size() * 3);

Iterator iter = subMap.values().iterator();

while(iter.hasNext()) {

// our collection here is actually a ConcurrentHashMap!

Map m = (Map) iter.next();

// this is not good as we are transversing the whole list twice!
// TODO: Create custom iterator for MultiSortedMap to avoid this?

Iterator iter2 = m.keySet().iterator();

while(iter2.hasNext()) {

users.add(iter2.next());
}
}

return users;
}
[/code]
[color=blue]NOW:[/color]
[code]
@Override
public Collection findUsers(int minAge, int maxAge) {

// Remember that our index is of the type MULTI_SORTED.

MultiSortedMap map = indexByAge.getMap();

// A multi-sorted-map will store a Set for each key.
// That's why our index is NON-UNIQUE meaning the same
// key (minAge, maxAge) can be associated with different entries in the map.
// The map is also sorted by its key, meaning we can
// perform range searches efficiently like below.

return map.subMapValues(new Key(minAge), true, new Key(maxAge), true);
}
[/code]

[b]Simpler and more efficient as you don't need to copy anything. [i]subMapValues[/i] returns a collection of users (backed up by a collection of sets) so you can iterate through all elements efficiently.[/b]

multi Index

hajmoh — GMT

Hello,
i create index for many attributes and i want to get a list which satisfies more than 1 constraints.
[code]
// indexing QUEUEID, SHIPPINGDONE, TIMENEXTSEND
if (!_indexManager.checkIndex(IX_QUEUESV4_GETMSG))
{
Index _index = new Index(IX_QUEUESV4_GETMSG, MAP_NAME, Index.TYPE.MULTI,
QueuesMessage.class, "queueId", "shippingDone", "timeNextSend");
boolean _ok = _indexManager.createIndex(_index, m_space4J);
// if (_ok)
// logger.info(_index + " created !");
// else
// logger.info("error while creating index" + _index);
}
[/code]
Below how we can get list satisfying one constraint.
[code]
List _messages = null;
MultiMap _multiMap = (MultiMap) m_indexGetMsg.getMap();
_messages = _multiMap.get(new Key(queueId));

return _messages;
[/code]
my question is how we can use multi elements ?, not just one as i did above [color=red]_multiMap.get(new Key(queueId))[/color] and what we can find on phonebook example.

thanks for responses.

Couple of questions regarding the product and object Recovery

Sue — GMT

Hi,

Our requirement is to cache certain ammount of data from database to improve the performance say as for maximum 3000 required users. When the request comes in for new user (beyond the max limit) to store in cache, we have to delete the oldest inactive user object from the memory and add the new user to cache. Is there a way to identify the most inactive object in space4j or do we need to maintain some sort of property.
The next question is regarding the recovery, as our requirement is to hold only maximum 3000 user records in cache, suppose from the begining of the system usage we cached in total 5000 user records but deleted 2000 of them while adding new records, when the service restarts how can we avoid the log command for the deleted objects?

From the recent forums, I seen that the library has been updated with fix to the issues reported recently, which version of jar do I have to download, is it stable version or beta version.

Thanks,
Sue

Inconsistency between CreateObjectCmd and RemoveCmd

Riduidel — GMT

Hi, there seems to be a little inconsistency in the application of Create/Remove commands to SimpleSpace4J.
Indeed, i've created a small test (in my mavenization process) to check application of basic commands to a SimpleSpace4J.

In this test, I've the following test method :

[code]

@Test
public void testStoreAndRetriveSimpleObject() throws CommandException, LoggerException {
Object test = new TestData();
String key = "testStoreAndRetriveSimpleObject.simpleObject";
CreateObjectCmd createObj = new CreateObjectCmd(key, test);
testedSpace4J.exec(createObj);
RemoveCmd removeObj = new RemoveCmd(key, null);
testedSpace4J.exec(removeObj);
}
[/code]

where testedSpace4J is a SimpleSpace4J.

When executing this test method, I obtain the following result :

[code]
org.space4j.CommandException: Target is not a collection or map: testStoreAndRetriveSimpleObject.simpleObject (org.space4j.implementation.SimpleSpace4JTest$TestData)
at org.space4j.command.RemoveCmd.execute(RemoveCmd.java:85)
at org.space4j.implementation.SimpleSpace4J.exec(SimpleSpace4J.java:149)
at org.space4j.implementation.SimpleSpace4J.exec(SimpleSpace4J.java:140)
at org.space4j.implementation.SimpleSpace4JTest.testStoreAndRetriveSimpleObject(SimpleSpace4JTest.java:48)
[end of trace is of no interest here]
[/code]
I don't understand the rationale behind this assymetric operation. Can someone explain it to me ? Thanks

To my mind, a logical way of handling things would be to have sylmetric create/delete operations, no ?

CreateCollectionCmd does not call map.check ?

Riduidel — GMT

Hi,
when all other Create*Cmd calls map.check before calling map.put, CreateCollectionCmd don't do so.
Is there any reason for that ?

maven/generics and other subjects

Riduidel — GMT

Hi, a long time ago, I worked a little on mavenification of space4j. Would you like I send a svn patch with all the modifications ?
In the same direction, I've notice that space4j is not yet supporting generics. Do you want I take a little look at that subject ?

indexed filtering on multiple bean properties

snacht — GMT

I have a web grid showing 250000 products.
the user can filter on of product properties
for example:
price > 30 AND
discount < 20 AND
type.contains("mx")

No problemo for sql. (even for type.contains('mx') -> oracle can create indexes on calculated values).
how about space4j?
Can I implement an index for this in space4j.
Would I have to create a separate index for each combination of bean properties?
Any help welcome.
Thanks,
Stefaan

Experience feedback and other questions

Jonathan — GMT

Hello.

I'm currently building a social website for a company that foresees a population of 500000 users in the system. The system will be hosted in the cloud by Amazon Web Service. According to my calculation the size of the "database" will grow to 8 GB. I'm really interested by Space4J to carry out the persistence. Do you know/have any experience feedback with this kind of system ?

I also have some technical questions :

Can index be built for a map that already have datas ? If yes, what is the expected behavior : access to index map is blocking ? Is there a way to be notified when the index is ready ?

When i get an object from Space4j is it a copy ? If no, how does it work when two threads want to update the same object ? What is the good practice ?

synchronous replication across the cluster supported ?

Thanks.

Jonathan