Pairwise forces between particles in cosmological N-body simulations are generally softened to avoid hard collisions. Physically, this softening corresponds to treating the particles as diffuse clouds rather than point masses. For particles of unequal mass (and hence unequal softening length), computing the softened force involves a nontrivial double integral over the volumes of the two particles. We show that Plummer force softening is consistent with this interpretation of softening while spline softening is not. We provide closed-form expressions and numerical implementation for pairwise gravitational force laws for pairs of particles of general softening scales $epsilon_1$ and $epsilon_2$ assuming the commonly used cloud profiles: NGP, CIC, TSC, and PQS. Similarly, we generalize Plummer force law into pairs of particles of general softenings. We relate our expressions to the gaussian, Plummer and spline force softenings known from literature. Our expressions allow possible inclusions of pointlike particles such as stars or supermassive black holes.

The GRACOS (GRAvitational COSmology) code, a parallel implementation of the particle-particle/particle-mesh (P3M) algorithm for distributed memory clusters, uses a hybrid method for both computation and domain decomposition. Long-range forces are computed using a Fourier transform gravity solver on a regular mesh; the mesh is distributed across parallel processes using a static one-dimensional slab domain decomposition. Short-range forces are computed by direct summation of close pairs; particles are distributed using a dynamic domain decomposition based on a space-filling Hilbert curve. A nearly-optimal method was devised to dynamically repartition the particle distribution so as to maintain load balance even for extremely inhomogeneous mass distributions. Tests using $800^3$ simulations on a 40-processor beowulf cluster showed good load balance and scalability up to 80 processes. There are limits on scalability imposed by communication and extreme clustering which may be removed by extending the algorithm to include adaptive mesh refinement.