Manipulating data

Manipulating data http://www.flickr.com/photos/maggiew/6145245962/

To manipulate data, you need a reference to the data… Welcome to Object keys • Full Key = app engine application ID + separator + entity kind + separator + entity group + separator + entity ID key • Actually, each part is hashed before concatenation • Typical ways to specify the entity ID key • You set it yourself as a String (or Long) • You let GAE automatically assign a Long • You let GAE automatically set an encoded String • You set it yourself as a custom Key object

Setting your own String as a key • package edu.oregonstate.mobilecloud.lectures.clouddatastore; • import javax.jdo.annotations.PersistenceCapable; • import javax.jdo.annotations.Persistent; • import javax.jdo.annotations.PrimaryKey; • @PersistenceCapable • public class Test2Simple { • @PrimaryKey • @Persistent • private String title; • public String getTitle() { • return title != null ? title : ""; • } • public void setTitle(String title) { • this.title = title; • } • } Declare @PrimaryKey Before trying to makePersistent, be sure to set the value Any existing entity of that type with that key is overwritten

Letting GAE automatically assign a Long key package edu.oregonstate.mobilecloud.lectures.clouddatastore; import javax.jdo.annotations.IdGeneratorStrategy; import javax.jdo.annotations.PersistenceCapable; import javax.jdo.annotations.Persistent; import javax.jdo.annotations.PrimaryKey; @PersistenceCapable public class Test3Members { @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) private Long id; public Long getID() { return id; } Declare @PrimaryKey Specify IDENTITY generator type Do not try to assign the ID when instantiating object

Listing all entities of a certain kind <%@ page import="javax.jdo.Query" %> <%@ page import="javax.jdo.PersistenceManager" %> <%@ page import="edu.oregonstate.mobilecloud.PMF" %> <%@ page import="edu.oregonstate.mobilecloud.lectures.clouddatastore.*" %> <%@ page import="java.util.*" %> <%@ page import="edu.oregonstate.mobilecloud.PMF" %> <% PersistenceManager pm = PMF.getPMF().getPersistenceManager(); try { Query query = pm.newQuery(Test3Members.class); List<Test3Members> allTest3 = (List<Test3Members>)query.execute(); for (Test3Members obj : allTest3) { out.write("<B>"+obj.getDepartment()+obj.getCoursenum() +": difficulty "+obj.getDifficulty()+"</B>"); out.write("<div style='height:100; “ +”overflow-y:auto'>"+obj.getDescription()+"</div>"); out.write("<BR><BR><BR>"); } } finally { pm.close(); } %> Create Query Execute Cast to List of objects test5.jsp

Retrieving a subset of entities based on a value <%@ page import="javax.jdo.Query" %> <%@ page import="javax.jdo.PersistenceManager" %> <%@ page import="edu.oregonstate.mobilecloud.PMF" %> <%@ page import="edu.oregonstate.mobilecloud.lectures.clouddatastore.*" %> <%@ page import="java.util.*" %> <%@ page import="edu.oregonstate.mobilecloud.PMF" %> <% PersistenceManager pm = PMF.getPMF().getPersistenceManager(); try { Query query = pm.newQuery(Test3Members.class, "department == :dd && coursenum >= :cn"); List<Test3Members> allTest3 = (List<Test3Members>)query.execute("CS", 400); for (Test3Members obj : allTest3) { out.write("<B>"+obj.getDepartment()+obj.getCoursenum()+": difficulty "+obj.getDifficulty()+"</B>"); out.write("<div style='height:100; overflow-y:auto'>"+obj.getDescription()+"</div>"); out.write("<BR><BR><BR>"); } query.closeAll(); } finally { pm.close(); } %> Create Query with parameters Bind parameter values on execute Cast to List of objects Close query when done test5a.jsp

Retrieving an object by key <%@ page import="javax.jdo.Query" %> <%@ page import="javax.jdo.PersistenceManager" %> <%@ page import="edu.oregonstate.mobilecloud.PMF" %> <%@ page import="edu.oregonstate.mobilecloud.lectures.clouddatastore.*" %> <%@ page import="java.util.*" %> <%@ page import="edu.oregonstate.mobilecloud.PMF" %> <% PersistenceManagerpm = PMF.getPMF().getPersistenceManager(); try { Query query = pm.newQuery(Test3Members.class); query.setOrdering("id desc"); List<Test3Members> allTest3 = (List<Test3Members>)query.execute(); long maxId = allTest3.get(0).getID(); Test3Members highest = pm.getObjectById(Test3Members.class, maxId); out.write(highest+"..."+highest.getID()); query.closeAll(); } finally { pm.close(); } %> Use getObjectById to retrieve If object might not exist, catch JDOObjectNotFoundException or JDOException test5b.jsp

Deleting entities <%@ page import="javax.jdo.Query" %> <%@ page import="javax.jdo.PersistenceManager" %> <%@ page import="edu.oregonstate.mobilecloud.PMF" %> <%@ page import="edu.oregonstate.mobilecloud.lectures.clouddatastore.*" %> <%@ page import="java.util.*" %> <%@ page import="edu.oregonstate.mobilecloud.PMF" %> <% PersistenceManager pm = PMF.getPMF().getPersistenceManager(); try { Query query = pm.newQuery(Test3Members.class, "coursenum >= 600"); List<Test3Members> toDelete = (List<Test3Members>)(query.execute()); pm.deletePersistentAll(toDelete); query.closeAll(); } finally { pm.close(); } %> Retrieve list of objects to delete (which seems inefficient?!?!) Invoke pm.deletePersistentAll test5c.jsp

By the way… • Query has a closeAll() method. • close the query when you’re done with results • Query has a deletePersistentAll() instance method. It is painfully slow in some SDKs. • If you use this method, be sure to assess how fast it is on very large sets of objects before you rely on it.

Now what if two people want to manipulate an object at the same time? <% String title = request.getParameter("title"); if (title == null) out.write("provide a title"); else { PersistenceManager pm = PMF.getPMF().getPersistenceManager(); try { Test1Object obj = null; try { obj = pm.getObjectById(Test1Object.class, title); } catch (JDOObjectNotFoundException doesNotExist) { obj = new Test1Object(); obj.setTitle(title); } obj.addCategory(Math.round(Math.rand()*100)); pm.makePersistent(obj); } finally { pm.close(); } } %> … @PersistenceCapable public class Test1Object { @PrimaryKey @Persistent private String title; … @Persistent private TreeSet<Integer> categories = new TreeSet<Integer>(); public void addCategory(Integer category) { categories.add(category); } }

You need a transaction when your servlet needs to… • Update an entity if it already exists, or create a new one otherwise • Compute a new member variable value based on an existing value • Modify multiple entities in a consistent way • … Do anything where your servlet could potentially be confused if gets two simultaneous requests/hits from users

Setting up a simple transaction Transaction trans = pm.currentTransaction(); trans.begin(); Test1Object obj = null; try { obj = (Test1Object)pm.getObjectById(Test1Object.class, title); } catch (JDOObjectNotFoundExceptiondoesNotExist) { obj = new Test1Object(); obj.setTitle(title); } pm.makePersistent(obj); trans.commit(); Tell the current transaction to begin tracking operations Do your operations Commit your changes (On uncaught exception, changes are rolled back automatically) test2.jsp

Transactions on GAE • A transaction places an optimistic lock all of the entities in a certain group • If any other transaction tries to modify entities in that group, then • The first transaction to commit is successful • The second (and later) transactions fail on commit • In practice, contention can cause commits to fail… so just retry • Very very similar to DBMS optimistic locking, which we all know and love, except for this concept of entity groups

Entity groups • Groups partition the set of all entities in GAE • Every entity is in exactly one group • Every group has at least one entity • Every group is very similar to tree • It has a root (which is used to identify the group) • Every entity has a parent (which for roots is the entity itself – this is where the tree analogy breaks) • Overall, therefore, the datastore is a forest

In other words… • A transaction can lock one, and only one, tree in the forest Univ Oregon Oregon State Art History Info Sys UnivCalif Berkeley CS ECE IS434 IS210 CS ECE CS361 CS496 CS364 CS364 CS351

Entities in a group are stored in the same place(s) in distributed file system • Remember: Your data values are copied (replicated) to many different servers. • Storing entities that are locked together (a group) helps to improve performance • Downside: an entity’s group cannot be changed after the entity is first saved • Odd side-effect: entities are only affected by a transaction if they are retrieved by implicit specification of a file system location, e.g., retrieval by key or via a query that references an ancestor filter (e.g., parent key)

How to set up entity groups • Option 1: Use JDO Relationships to specify that the child is dependent on the parent • Option 2: Use custom JDO Key object that identifies the parent entity for a child entity

Option 1: Use JDO Relationships • This is different than a compound object (an object that has multiple parts) • It is when you have two entities, one of which is a child of another • The parent “owns” the child in JDO-speak

Example code snippet • … • @PersistenceCapable • public class Test3JDOEmployer { • @PrimaryKey • @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) • @Extension(vendorName = "datanucleus", key = "gae.encoded-pk", value = "true") • private String id; • @Element(dependent="true") • private HashSet<Test3JDOEmployee> employees = new HashSet<Test3JDOEmployee>(); • … • @PersistenceCapable(identityType = IdentityType.APPLICATION) • public class Test3JDOEmployee { • @PrimaryKey • @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) • @Extension(vendorName = "datanucleus", key = "gae.encoded-pk", value = "true") • private String id;

Example servlet code Test3JDOEmployer boss = new Test3JDOEmployer(); boss.setName("Employer "+empname); for (intempid = 1; empid <= 3; empid++) { Test3JDOEmployee worker = new Test3JDOEmployee(); worker.setName("Employee "+empname); boss.addEmployee(worker); } pm.makePersistent(boss);

In the servlet… • In the servlet • Just initialize and store the parent (no explicit child save) • On load, just load the parent and access child via the parent as needed • FYI: Child objects are not loaded until they are read or written, so loading the parent can be faster.

Option 2: Use explicit JDO Keys • Which I’m not going to show you in detail because it is much more hassle to set up, and it doesn’t really add any more benefits. • But here’s how you can do it: • Only list the keys of the child in the parent • When instantiating the child, pass the parent’s key • In the child’s constructor, use KeyFactory to construct a child key, based on the parent’s key • See GAE Book 324-325 for details (note typos)

Summary of important points • Use a transaction if two simultaneous hits to the server could lead to inconsistency • Each transaction can only operate on a single entity group • To put multiple entities in the same group, use a JDO parent-child relationship • If you need a transaction and you need to modify multiple entity groups, use cursors and transactional tasks

Manipulating data