Sunday, August 10, 2008

c:forEach with JSF could ruin your day

My blog has moved, please update your links. You can find this article here

Last week, a co-worker of mine at Oracle was attempting to use a forEach tag to produce components in a loop and developed issues. Having some experience trying to deal with for each I started looking into the problem and found a disturbing problem.

The JSTL <c:forEach/> tag was written to make JSP development easier, but was never intended to be used with Java Server Faces. In JSP 2.1 some enhancements were made to improve the usage of non-JSF JSP tags intermingled with JSF by introducing deferred expressions. The idea behind the deferred expressions was to be able to create expressions that would normally have to be evaluated at JSP tag execution and allow them to be executed with JSF components instead. The problem is that the implementation of deferred expressions is flawed for the <c:forEach/> loop except for basic usage only.

Use Case

For the illustration of the problem with the <c:forEach/> I will simplify the use case. The are two main problems with the tag both of which involve the changing of the list bound to the items attribute. For the first, consider the following requirements:

  • For each loop required, JSF stamped iteration components cannot be used (like JSP tags that include a page)
  • Ajax used to partially render the page
  • Id's of components must be tied to the item in the loop to allow actions

Problem 1 - static EL

A simple example can be used to show the problem:

<h:form>
  <c:forEach var="item" items="#{bean.items}">
    <h:commandButton
      id="button_${var.key}"
      value="Remove #{var.key}"
      actionListener="#{bean.remove}" />
  </c:forEach>
</h:form>

Take for example a list of "A", "B", "C" and "D". The buttons render as:

ID Text
button_A Remove A
button_B Remove B
button_C Remove C
button_D Remove D

Now should the user click "Remove A", the managed bean removes the value from the items list. The result is:

ID Text
button_B Remove C
button_C Remove D
button_D Remove

Notice the mismatch of ID and text? Notice that the last button's #{var.key} evaluated to null?

To understand the problem, it is best to describe a step by step process of how the code works:

  1. The for each tag is executed and the first iteration (count 0) is begun
  2. The tag creates an IndexedValueExpression. This sub-class of ValueExpression.
    • The expression to get the items is stored (#{bean.items})
    • The index of the for each iteration is stored (0)
  3. This IndexedValueExpression is then added to the VariableMapper as the var - item.
  4. The body of the tag is executed
  5. The command button tag is instructed to create or find a component with ID button_A.
  6. The command button component is created with all its attributes.
    • When the component is created, the ValueExpression instances are created
    • Each ValueExpression is created with a reference to the FunctionMapper and VariableMapper
  7. The for each tag proceeds with its execution and processes indexes 1-3

When the action executes and removes A, the following occurs during the render phase. First, the for each tag looks for the button for B and finds it. Then the loop repeats. At the end of processing the tags, the code realizes that button_A was never referenced, and is therefore removed from its parent, the form.

The problem is that once the component is created, the ValueExpression instances are created, and are never re-created. This is because the expressions with their mappers are serialized and restored as part of the component state. This means that button_B which was found for the first item after item A was removed still has its original ValueExpression for the value attribute. This expression has the variable mapper with the indexed value expression for the var which has index 1.

This means that if components are used with IDs that tie to the items, it is possible to have the value expressions pointing to the wrong index in the for each loop once changes are made to the items list.

Problem 2 - Component State

One workaround to problem one is to use dynamically created IDs. This will violate our above requirement of needing to match the component to the item for advanced PPR (AJAX) replacement, but it is a consideration.

Lets use the above use case with the IDs removed:

<h:form>
  <c:forEach var="item" items="#{bean.items}">
    <h:commandButton
      value="Remove #{var.key}"
      actionListener="#{bean.remove}" />
  </c:forEach>
</h:form>

This renders as:

ID Text
j_id_id2 Remove A
j_id_id2j_id_1 Remove B
j_id_id2j_id_2 Remove C
j_id_id2j_id_3 Remove D

When A is removed, since the item key is not used in the ID, the above problem is not witnessed:

ID Text
j_id_id2 Remove B
j_id_id2j_id_1 Remove C
j_id_id2j_id_2 Remove D

What you will notice is that item B which used component with ID j_id_id2j_id_1 is now rendered by component with ID j_id_id2. Unlike above, this component has the correct index stored for it, but its state may break the view. Now command buttons are not a great example of this, but many components have state that is more evident, like tree node expansion. What this will mean to the user is that component state that belongs with item B is now associated with item C.

How could this be addressed?

The fundamental problem is that the implementation of the for each loop tag assumes that the location of the components that are created are independent of the items used to produce them. The ValueExpression instances are created for components with hard coded indexes and there is no way that I know of to update the index to the correct value later.

One fix is to allow the for each loop to use keys for the item instead of indexes. The obvious problem with this idea is that it would involve a significant API enhancement to be able to specify a key for an object and the use of a Map instead of a List or array to be able to lookup the value for the key. Something like:

<h:form>
  <c:forEach var="item" keys="#{items.keys} items="#{bean.items}">
    <h:commandButton
      value="Remove #{var.key}"
      actionListener="#{bean.remove}" />
  </c:forEach>
</h:form>

Where keys would be a List or array of serializable objects for keys in the items which would be represented as a Map.

Summary

As I have always recommended on the Apache MyFaces and Facelets mailing lists the best solution is to never use JSTL tags for JSF development. Unfortunately there are some pieces of functionality which is hard to replicate using components. Perhaps JSF 2.0 will allow things like component controlled sub-page inclusion.

Saturday, June 07, 2008

Understanding the dataTable

My blog has moved, please update your links. You can find this article here

The <h:dataTable/>, although easy for many to understand to use is often not understood in how it actually works. Since my article on component tree construction and rendering with respect to JSTL seemed to be received pretty well, I have decided to cover how the <h:dataTable/> works.

Table Of Contents

Looping components

The dataTable is one of many components that loop, although it is unfortunately the only one provided with core JSF. Other looping components are from 3rd party libraries. Here are some:

Each of these components, although renders quite different HTML all work using a similar design, modeled by the core <h:dataTable/>.

General Architecture

The API is simple, but the architecture, not so much. The API involves the construction of a set of components that produce an outer HTML element followed by inner HTML elements. For example, the <h:dataTable/> creates an outer <TABLE> with a <TR/> for each iteration of the loop. Others like <t:dataList>, <ui:repeat> and <tr:iterator> produce no HTML usually, so they simply allow one to loop over a set of components for many objects in a model.

Each of these components have value and var attributes, although their names may change slightly from component to component. The value provides some kind of model data to the component. This may be a model class (like a tree model for a tree component), or just object arrays or collections of objects. Each component differs on what values they support, but either way all point to a model of data that contains many objects. When the component is rendered, each object in the model is iterated over. In order for page developers to access the data, an EL variable is injected into scope temporarily. This variable has a name given by the var attribute. Take for example this code that shows all of the cookies sent to the server in a table in a servlet environment:

<h:dataTable
  var="_cookie"
  value="#{facesContext.externalContext.request.cookies}">
  <h:column>
    <f:facet name="header">
      <h:outputText value="Name" />
    </f:facet>
    <h:outputText value="#{_cookie.name}" />
  </h:column>
  <h:column>
    <f:facet name="header">
      <h:outputText value="Value" />
    </f:facet>
    <h:outputText value="#{_cookie.value}" />
  </h:column>
</h:dataTable>

The value is an array of Cookie objects. The var is "_cookie" (I use an underscore to make sure the name does not intefere with any scoped objects). The table renders the header and footer as a normal JSF component, but then iterates over the data. In this case, it converts the array to a DataModel. It then sets the rowIndex which caused the rowData to return the current cookie (so a row index of 0 returns the first cookie and setting the index to 1 will cause the 2nd cookie to be returned). Basically this is a simple iterator pattern. Some components loop through all, some loop only through a subset (the table can loop through a portion using the rows and first attributes).

The most important thing to note is that even though there may be several cookies, there are only ever 4 outputText components (1 in each header and one in each column).

Encoding the Children

What happens to produce multiple <TR/> elements is that the TableRenderer encodes its children in a loop. Here are the steps of TableRenderer.encodeChildren:

  1. get the first row to be rendered from UIData.getFirst()
  2. get the number of rows to be rendered from UIData.getRows()
  3. write the <TBODY/> start tag
  4. loop until the correct number of rows have been rendered
    1. if row is not available then stop
    2. render the row, encoding all the children recursively

Understanding "var"

The var is not controlled by the renderer, but the component, UIData to be precise. Non-UIData components that loop perform similar logic if designed after the JSF table. The scope of the var is limited to when the <t:dataTable/> is currently iterating. To be exact, it is available when the rowIndex is not -1. In the UIData.setRowIndex(int) method, if the index is being set to a value not equal to -1, then the row state is updated for the appropriate index. If the value is -1, the state is cleared.

The state is comprised of all the children components that implement EditableValueHolder. So before the row index is changed, UIData saves the value, localValueSet, valid and submittedValue properties for each child below the table and restores any state saved for the new index. This means that only EditableValueHolder components may vary state for different rows. Non-EditableValueHolder components may only vary their behavior (a.k.a. attributes) per row by using EL expressions that use a value that changes per row, like var.

The code stores the row data for the current row into the request map using the value of var as the key. When the row index is set to -1, the key and value are removed from the request map. This means that if the var is the same as another request map variable, it will be overridden and then lost. Therefore, it is very important to use a value for var that will not conflict with other request map variables. This is why I always prepend my var values with and underscore.

Note:
The var is only in scope when the UIData's row index is not -1. This means that it is not available when the component tree is being built, or usually when accessed from outside the component. If you would like to get access a child component of UIData with the correct scope, you must set the row index to the desired value, otherwise the data will not be there.

Other Phases

The decode, validate and updating processes of the <h:dataTable/> work the same way as the rendering. They loop through each row index and invoke the appropriate methods on the children components. Since the code to alter the var and save and restore the states is in UIData, the behavior is shared.

Understanding Client IDs

You may have noticed that the client IDs of looped components are altered in a table. Take this use case:

<h:dataTable
  id="cookies"
  var="_cookie"
  value="#{facesContext.externalContext.request.cookies}">
  <h:column>
    <h:outputText id="cookieName" value="#{_cookie.name}" />
  </h:column>
</h:dataTable>
<h:outputText id="after" value="After the table" />

The two <h:outputText/> components will have different client IDs. The one outside the table will just be "after" (assuming that there are no parent naming containers). The one in the table however will be "cookies:0:cookieName". The number, '0' in this example, will be the row index. This happens because the UIData component returns a different ID from getClientId when the row index is not -1. The code from the JSF RI (a.k.a. Mojarra) is:

    @Override
    public String getClientId(FacesContext context)
    {
        String clientId = super.getClientId(context);
        int rowIndex = getRowIndex();
        if (rowIndex == -1)
        {
            return clientId;
        }
        return clientId + NamingContainer.SEPARATOR_CHAR + rowIndex;
    }

As you can see, NamingContainer.SEPARATOR_CHAR + rowIndex is added onto the end of the table's client ID when the row index is not -1. This ID convention stops ID collision in the HTML, but it also is used for event broadcasting.

Event Broadcasting

When a child of the table is decoded, it decodes correctly due to the client ID. Take for example an <h:inputText id="myInputText"/> component that is in a table. Using just two rows as an example here is a map of the client IDs:

Row Index Client ID
-1 myTable:myInputText
0 myTable:0:myInputText
1 myTable:1:myInputText

When the renderer for the input text is decoding, it looks for a value submitted by the client using the client ID as the key. Since the client ID changes, the input text correctly decodes the value from the client for the current row index.

Now this is great for decoding, but a different mechanism must be used for the queuing and the broadcasting of events. The UIData component wraps all events in a wrapper class that stores the event being queued, the client ID of the UIData component and the current row index. Here is what each property is used for:

Client ID
The client ID is used in case this table is nested in another table. By identifying the event to broadcast using this ID, the framework can ensure events are broadcast to the correct state of a nested looping component.
Event
The original event that will be unwrapped during broadcast. The wrapper delegates much of its functionality to the wrapped event (like the phase ID).
Row Index
During broadcasting of the event, this stored index is used to correctly re-position the model to the correct row so that the event is fired in the same row state as when it was queued.

Binding

When binding looping components or children of them to managed beans, it is important to realize what the row index is. For example, if you have an <h:commandLink action=#{bean.actionMethod}"/> component as a child of a column in a table, the action method is run in the scope of the correct row index. This is because the event is wrapped knowing the correct row index. Therefore, your action method may access the current row data.

Also since the var's scope is only valid during iteration, it is not accessible during the construction of the component tree. Therefore and JSP tags or Facelets TagHandlers may not attempt to retrieve row data using the UIData APIs.

Conclusion

The data table component and looping components like it re-use a set of components encoded several times, once for each item to be rendered. The value, submitted value, if the component is valid and if the local value is set properties of EditableValueHolder children components are supported across iteration of the model. Other properties of components are not saved and thus must use EL to vary their behavior based on the current row.

Client IDs and event wrappers are used to ensure that the event model of JSF is not broken by the iteration of the component.

Hopefully this helps you understand looping components better and helps you design better pages as a result. Thank you for reading.

Tuesday, March 18, 2008

Build time vs. render time

My blog has moved, please update your links. You can find this article here

Overview

There always seems to be someone posting about how MyFaces or JSF is "broken" and it turns out that they misused JSTL tags. No matter how many times a solution is posted, the issue comes up again. Therefore, I am writing this blog to help those that do not understand how facelets and JSP view handlers work.

The Problem

JSF works entirely differently than JSP. JSP, when used for JSF, is used to build a JSF component tree, not render a page. JSTL tags alter the contents of what is built, the component tree, not the HTML that is output by the component renderers.

A Background of JSP

JSP was written so that authoring servlets would be easier. JSP pages are servlets, nothing more. Using a JSP compiler, a JSP file is compiled into a Java Servlet. Before tag libraries were introduced, JSP pages simply were compiled down into System.out.println(...); statements.

Tag libraries enhanced JSP by providing an API to tag authors to write Java code that would not have to be embedded in <% ... %> tags. The tags gave developers access to the text content of the tag body so that they could alter it or use it in a custom way. After some time, Sun released the Java standard tag library API, also known as the JSTL.

The JSTL

The JSTL is a set of JSP tags to perform common JSP functionality (note that they are not related to JSF in any way at all). Many of them perform logic to alter their contents. In the case of c:if and c:choose, they choose which tag contents to be included in the response or not.

I will be mainly talking about the JSTL core tags in this article (catch, choose, forEach, forTokens, if, etc.). It is these that have such a profound, and often confusing to some people, affect on JSF.

What JSF Is

JSF is a component technology, and architecturally is not related to JSP at all. In fact, using JSP with JSF has drawbacks, included serious performance implications due to the architecture of JSP and how the JSP view handler works.

Note

Sun probably made the default view handler as one that uses JSP, so they would not have to admit that JSP did not meet the needs of users and needed to be replaced.

Component Technology

What I mean by JSF being a component technology, is that HTML is produced by the processing of a component tree by renderers. This design is much more similar to Swing than it is Servlets and JSP. Regardless of the view handler that is used, a tree of components is built.

Note

A component is simply a Java Object that implements the UIComponent interface, nothing more.

These components simply store attributes, have some behaviors and are similar in nature to the Swing Tree component. JSF relies on renderers (classes that extend Renderer) to produce content from the component tree. During the restore view, a JSF view should be restored to its previous state, so that it appears to code that there was never a round trip for the code to the client (this is important as I discuss the JSTL tags and especially c:forEach).

Note

Components do not have to use renderers to produce their content, but it is considered bad design to use the encode methods in a component to render a response.

Note

JSF is typically used to produce HTML, but can be used to generate any output that doesn't even have to be XML related, although the ResponseWriter's API is designed for XML content.

JSF with JSP

JSF is implemented using JSP by default (see my note above why this stinks). What this means is that JSP is used to build the JSF component tree. This is important to note because JSP is not used to generate any of the HTML, unlike a typical JSP page. There are two main stages of JSF on JSP:

  1. Building of the component tree using JSP tags
  2. Rendering of the component tree using the standard JSF lifecycle

Building of the component tree using JSP tags

As I mentioned earlier, JSF is a component technology. When JSP tags are used, instead of writing text (HTML) onto the servlet response, the tags create components and set attribute values on them. Most of this work is done by UIComponentClassicTagBase. The class hierarchy is:

  • java.lang.Object
    • javax.faces.webapp.UIComponentTagBase
      • javax.faces.webapp.UIComponentClassicTagBase
        • javax.faces.webapp.UIComponentELTag
        • javax.faces.webapp.UIComponentTag

The doStartTag method of UIComponentClassicTagBase calls findComponent, which despite its name creates the component (if needed) and calls setProperties. Therefore, the JSF component is created when the start of the JSP tag is processed. The setProperties method is responsible for setting the attributes of the component. Typically there is a one-to-one map of JSP tag attribute to JSF component attribute, so the setProperties simply copies the attributes over, doing any necessary conversion of the string value in the JSP attribute to an object that the component expects for the attribute.

How JSTL fits in

Most of the "work" of a component takes place during rendering. For example, the h:dataTable sets up the var attribute during rendering (and other phases too, but that is not pertinent to this article). This means that EL expressions that rely on variables that only have scope during the rendering phase are not valid during component creation. That is to say, there is no component lifecycle method for when the component is created.

Since JSTL tags are plain JSP tags and do not involve the JSF lifecycle, they do not have access to the environment that components create, like the var variable of an h:dataTable. Take this code for example:

<f:view> <h:dataTable var="_row" value="#{bean.rows}> <h:column> <c:choose> <c:when test="#{_row.editable}"> <h:inputText value="#{_row.value}" /> </c:when> <c:otherwise> <h:outputText value="#{_row.value}" /> </c:otherwise> </c:choose> </h:column> </h:dataTable> </f:view>

Although from the pure XML standpoint this looks valid, it is not. Thinking about what I just said it is a simple problem:

  1. Data table component created from the data table JSP tag
  2. Column component created from its JSP tag
  3. The when tag in the choose attempts to evaluate #{_row.editable}
    • Since _row is not bound (yet) in the EL resolver, null is returned
      (an argument could be made that the EL engine should throw an error or at least give a warning, but that is not how it is designed)
    • Since null is technically false, the outputText is created instead of the inputText
  4. The output text component is created by its tag

So the page author was probably expecting that the choose would be evaluated for every row in the table. But the table has no rows at this point, the component is being created. So, the inputText component is never created as the when tag will not process its contents if the test fails.

The proper solution is to use JSF functionality and not JSP functionality. To the confusion of JSP developers, there are no c:if, c:choose or c:forEach equivalent components (see the Tomahawk Sandbox limitRendered for c:if and c:choose functionality and the Tomahawk dataList or Trinidad iterator components for c:forEach type of functionality). Without third party components, the rendered attribute can also work, although it requires more work:

<f:view> <h:dataTable var="_row" value="#{bean.rows}> <h:column> <h:inputText value="#{_row.value}" rendered="#{_row.editable}" /> <h:outputText value="#{_row.value}" rendered="#{not _row.editable}" /> </h:column> </h:dataTable> </f:view>

In this case both the input and the output components are created. The rendered is evaluated by the renderer during the rendering phase while the data table is iterating over each of its rows.

When it is okay to use JSP tags with JSF

So now that you know the problem, you may ask why is using any non-JSF, JSP tags allowed? This is because they still work, but must be used correctly. Since JSP tags are evaluated while components are being built, they can control which components to create. For example, I may want to include certain components in a page template if the current user is an administrator. Because the components are not created if the user is not an administrator, there will be less components objects created and less components to process for each JSF lifecycle phase, and thus improving performance.

It is very important to remember that you cannot have components "re-appear" on post back of a JSF form. This is because a JSF component tree should never be altered between having its state saved and when its state is restored. This is very important, so let me say again, a JSF component tree should never be altered between having its state saved and when its state is restored. The reason is that this would violate the JSF specification/design. Take for example a c:forEach loop that when evaluated had five items it its list. Assuming that there was one JSF component tag inside the for each tag, that means five components were created. Now JSF saves the state for five components. Then lets say this for each loop data is coming from a database that can change and now there are only four items. When JSF attempts to restore the component tree, there is data for 5 components, but there are only 4 in the component tree. This causes a restore state exception.

Therefore, always ensure that when a component tree is restored, it is not changed! There is no problem changing a component tree after restoring it or before saving it, only between those times.

Hey, what about Facelets?

I did mention that I would talk about facelets, didn't I?

Facelets is similar in its behavior to JSP although it has much more efficient code and is designed quite differently than JSP. Instead of JSP tags, facelets has TagHandlers. These tag handlers analyze the XML content of a facelet and create components. Just like JSP tags, the tag handlers have no access to the render-time scope of the components. In fact, facelets provides tag handlers to mimic the JSTL functionality. These tag handlers have the same pitfalls as the JSP tags. Therefore, the same considerations and rules apply to tag handlers that apply to JSP tags.

Other Blogs

Here are some other helpful resources on this topic: