Skip to content

Pagination

Dynorm provides cursor-based pagination for efficient traversal of large result sets.

Basic Pagination

// Get first page (10 items)
page, err := UserRepo.
    Where("Status", "=", "active").
    Paginate(10, "")

// Process items
for _, user := range page.Items {
    fmt.Printf("User: %s\n", user.Email)
}

// Check for more pages
if page.HasMore() {
    cursor := page.NextCursor()
    // Use cursor for next page
}

Page Object

The Paginate method returns a page with:

type Page[T any] struct {
    Items []*T      // Entities in this page
    Count int       // Number of items
}

// Methods
page.HasMore()      // true if more pages exist
page.NextCursor()   // Cursor for next page
page.ScannedCount() // Total items scanned (before filtering)
page.IsEmpty()      // true if no items
page.First()        // First item in page
page.Last()         // Last item in page

Cursor-Based Navigation

func GetAllActiveUsers() ([]*User, error) {
    var allUsers []*User
    var cursor dynorm.Cursor = "" // Start with empty cursor

    for {
        page, err := UserRepo.
            Where("Status", "=", "active").
            Paginate(100, cursor)

        if err != nil {
            return nil, err
        }

        allUsers = append(allUsers, page.Items...)

        if !page.HasMore() {
            break
        }

        cursor = page.NextCursor()
    }

    return allUsers, nil
}

Iterator Pattern

For streaming large result sets:

iter := UserRepo.
    Where("Status", "=", "active").
    Iterate(100) // Page size of 100

defer iter.Close()

for iter.Next() {
    user := iter.Item()
    fmt.Printf("Processing: %s\n", user.Email)
}

if err := iter.Err(); err != nil {
    return err
}

Iterator Methods

// Advance to next item
hasNext := iter.Next()

// Get current item
item := iter.Item()

// Check for errors
err := iter.Err()

// Release resources
iter.Close()

// Collect all remaining items
items, err := iter.Collect()

// Process each item with callback
err := iter.ForEach(func(user *User) error {
    return processUser(user)
})

API Endpoint Example

func HandleListUsers(w http.ResponseWriter, r *http.Request) {
    cursor := r.URL.Query().Get("cursor")
    pageSize := 20

    page, err := UserRepo.
        Where("Status", "=", "active").
        OrderBy("CreatedAt", "desc").
        Paginate(pageSize, dynorm.Cursor(cursor))

    if err != nil {
        http.Error(w, err.Error(), 500)
        return
    }

    response := map[string]any{
        "users":   page.Items,
        "count":   page.Count,
        "hasMore": page.HasMore(),
    }

    if page.HasMore() {
        response["nextCursor"] = string(page.NextCursor())
    }

    json.NewEncoder(w).Encode(response)
}

Lambda Handler Example

type ListRequest struct {
    Status   string `json:"status"`
    PageSize int    `json:"pageSize"`
    Cursor   string `json:"cursor"`
}

type ListResponse struct {
    Users      []*User `json:"users"`
    NextCursor string  `json:"nextCursor,omitempty"`
    HasMore    bool    `json:"hasMore"`
}

func handler(ctx context.Context, req ListRequest) (ListResponse, error) {
    dynorm.SetContext(ctx)

    if req.PageSize == 0 {
        req.PageSize = 20
    }

    page, err := UserRepo.
        Where("Status", "=", req.Status).
        Paginate(req.PageSize, dynorm.Cursor(req.Cursor))

    if err != nil {
        return ListResponse{}, err
    }

    resp := ListResponse{
        Users:   page.Items,
        HasMore: page.HasMore(),
    }

    if page.HasMore() {
        resp.NextCursor = string(page.NextCursor())
    }

    return resp, nil
}

Cursor Details

Cursors are opaque base64-encoded strings containing the DynamoDB LastEvaluatedKey:

// Cursor looks like:
// "eyJpZCI6eyJTIjoiMDFIUTNLLi4uIn19"

// Don't parse or construct cursors manually
// Always use page.NextCursor()

Cursor Validity

Cursors are tied to the query conditions. Don't reuse cursors across different queries.

Combining with Filters

Pagination works with all query conditions:

page, err := UserRepo.
    Select("ID", "Email", "FirstName").
    Where("Status", "=", "active").
    Where("Role", "=", "admin").
    OrderBy("CreatedAt", "desc").
    Paginate(25, cursor)

Metadata

Access scan information:

page, _ := UserRepo.
    Where("Status", "=", "active").
    Paginate(10, "")

fmt.Printf("Items returned: %d\n", page.Count)
fmt.Printf("Items scanned: %d\n", page.ScannedCount())
fmt.Printf("Has more: %v\n", page.HasMore())

ScannedCount

ScannedCount() shows items scanned before filter expressions are applied. A large difference between scanned and returned items indicates an inefficient query.

Best Practices

Do

  • Use reasonable page sizes (20-100 items)
  • Return cursor in API responses for client pagination
  • Use iterators for batch processing
  • Include query conditions with pagination

Don't

  • Use very large page sizes (impacts latency)
  • Parse or modify cursor values
  • Reuse cursors across different queries
  • Ignore ScannedCount vs Count ratio

Flow Diagram

sequenceDiagram
    participant Client
    participant API
    participant Dynorm
    participant DynamoDB

    Client->>API: GET /users
    API->>Dynorm: Paginate(20, "")
    Dynorm->>DynamoDB: Query(Limit: 20)
    DynamoDB-->>Dynorm: Items + LastEvaluatedKey
    Dynorm-->>API: Page{Items, Cursor}
    API-->>Client: {users, nextCursor}

    Client->>API: GET /users?cursor=xxx
    API->>Dynorm: Paginate(20, cursor)
    Dynorm->>DynamoDB: Query(Limit: 20, ExclusiveStartKey)
    DynamoDB-->>Dynorm: Items + LastEvaluatedKey
    Dynorm-->>API: Page{Items, Cursor}
    API-->>Client: {users, nextCursor}

Next Steps