What are the key principles for designing a scalable software?

Picking up right software architecture can lay a scalable foundation for your application. For example, Microservices Architecture of software development can help in building a highly scalable & distributed system. Essentially, microservice architecture is a method of developing software applications as a suite of independently deployable, small, modular services in which each service runs a unique process and communicates through a well-defined, lightweight mechanism to serve a business goal.

Stateless design using REST can help achieve scalability wherever possible. In such application, minimal session elements need to be replicated while distributing the application over multiple hosts. Users can save their favorite URLs and thus there should be no need for the page flow, if we use REST.

Removing centralized bottlenecks from your software can help increasing the scalability capabilities. For example, instead of using DB backed authentication mechanism, you can use JWT (JSON Web Tokens) for authentication which does not require token to be checked at database level (centralized), instead each microservice can check token using public key (cryptography).

Logging can be done asynchronously to save precious time of a method call.

More processes vs more threads can be configured based on the demand of the target application. Generally it is advised to have a JVM with up to 2 GB memory because increasing memory beyond 2 GB incurs heavy GC pauses, and if we require more processing then we prefer to have a separate process for the JVM altogether. Multiple independent tasks should be run in parallel. Tasks can be partitioned to improve the performance.

If we improve upon the concurrency of the software piece, then we can increase its scalability. This can be achieved by reducing the dependency on the shared resources. We should try utilizing the latest hardware optimization through JAVA as much as possible. For example we can use Atomic utilities provided in java.util.concurrent.atomic package, or Fork & Join to achieve higher throughput in concurrent applications. We should try holding the shared locks for as little time as possible.

Resource pooling and caching can be used to improve the processing time. Executing jobs in batches can further improve the performance.

Picking up appropriate algorithm and data structure for a given scenario can help optimize the processing.

If we are using SQL in our application then we should tune the SQL, use batching whereever possible and create indexes on the essentials table columns for faster retrievals.

We should tune our JVM for optimum memory settings (Heap, PermGen, etc) and Garbage collection settings. For example if we do lot of text processing in our application with big temporary objects being created, then we should have larger Young Generation defined so that frequent gc run does not happen.

Keep up to date with new technologies for performance benefits.