Monday, October 27, 2014

Deploying Applications or Libraries to WebLogic Server Using Command Line

In this post, I will list the commands required to deploy apps/libraries to WebLogic Server

1.$ source $ML_HOME/server/bin/setWLSEnv.sh

Deploy Library:


1.$ java weblogic.Deployer -nostage -deploy -library \
2.-adminurl localhost:7001 \
3.-username weblogic -password my_secret \
4.-targets myserver \
5.my_shared_lib.war

Deploy Application:


1.$ java weblogic.Deployer -nostage -deploy \
2.-adminurl localhost:7001 \
3.-username weblogic -password my_secret \
4.-targets myserver \
5.-name myapp.war myapp.war

For development, you will use the "-nostage" mode which means that to deploy the app or library directly from the file system.
Any changes to that file location and a reload from WLS will take effect immediately.

undepoy app or lib

1.$ java weblogic.Deployer -undeploy \
2.-adminurl localhost:7001 \
3.-username weblogic -password my_secret \
4.-targets myserver \
5.-name myapp_or_lib.war
Posted by at
Labels: ,

Spring Web App

1. Overview

In this article I am going to illustrate how to implement a simple Login Page with Spring MVC for an application that’s handling the authentication with Spring Security in the backend.

2. The Login Page

Let’s start by defining a very simple login page:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
<html>
<head></head>
<body>
   <h1>Login</h1>
   <form name='f' action="j_spring_security_check" method='POST'>
      <table>
         <tr>
            <td>User:</td>
            <td><input type='text' name='j_username' value=''></td>
         </tr>
         <tr>
            <td>Password:</td>
            <td><input type='password' name='j_password' /></td>
         </tr>
         <tr>
            <td><input name="submit" type="submit" value="submit" /></td>
         </tr>
      </table>
  </form>
</body>
</html>
Now, let’s include a client side check to make sure the username and password have been entered before we even submit the form. For this example we’ll use plain Javascript, but JQuery is an excellent option as well:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
<script type="text/javascript">
function validate() {
    if (document.f.j_username.value == "" && document.f.j_password.value == "") {
        alert("Username and password are required");
        document.f.j_username.focus();
        return false;
    }
    if (document.f.j_username.value == "") {
        alert("Username is required");
        document.f.j_username.focus();
        return false;
    }
    if (document.f.j_password.value == "") {
   alert("Password is required");
   document.f.j_password.focus();
        return false;
    }
}
</script>
As you can see, we simply check if the username or password fields are empty; if they are – a javascript message box will pop up with the corresponding message.

3. Message Localization

Next – let’s localize the messages we’re using on the front end. There are types of such messages, and each is localized in a different manner:
  1. Messages generated before the form is processed by Spring’s controllers or handlers. These messages ca be referenced in the JSP pages and are localized with Jsp/Jslt localization (see Section 4.3.)
  2. Messages that are localized once a page has been submitted for processing by Spring (after submitting the login form); these messages are localized using Spring MVC localization (See Section 4.2.)

3.1. The message.properties files

In either case, we need to create a message.properties file for each language we want to support; the names of the files should follow this convention: messages_[localeCode].properties.
For example, if we want to support English and Spanish error messages we would have the file: messages_en.properties and messages_es_ES.properties. Note that, for English – messages.properties is also valid.
We’re going to place these two files in the project’s classpath (src/main/resources). The files simply contain the error codes and messages we need to display in different languages – for example:

1
2
3
4
5
6
7
message.username=Username required
message.password=Password required
message.unauth=Unauthorized access!!
message.badCredentials=Invalid username or password
message.sessionExpired=Session timed out
message.logoutError=Sorry, error login out
message.logoutSucc=You logged out successfully

3.2. Configuring Spring MVC Localization

Spring MVC provides a LocaleResolver that works in conjunction with its LocaleChangeInterceptor API to make possible the display of messages in different languages, depending on the locale setting. To configure localization – we need to define the following beans in our MVC configuration:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
@Override
public void addInterceptors(InterceptorRegistry registry) {
    LocaleChangeInterceptor localeChangeInterceptor = new LocaleChangeInterceptor();
    localeChangeInterceptor.setParamName("lang");
    registry.addInterceptor(localeChangeInterceptor);
}
@Bean
public LocaleResolver localeResolver() {
    CookieLocaleResolver cookieLocaleResolver = new CookieLocaleResolver();
    return cookieLocaleResolver;
}
@Bean
public MessageSource messageSource() {
    ReloadableResourceBundleMessageSource messageSource =
      new ReloadableResourceBundleMessageSource();
    messageSource.setBasename("classpath:messages");
    messageSource.setUseCodeAsDefaultMessage(true);
    messageSource.setDefaultEncoding("UTF-8");
    messageSource.setCacheSeconds(0);
    return messageSource;
}
One important and more advanced note here is that we need to make sure that the messageSource bean is defined in the right Spring context – the context that it’s going to be used in. Keep in mind that the web app has the root context and then it has one (or more) servlet contexts.
By default, the locale resolver will obtain the locale code from the HTTP header. To force a default locale, we need to set it on the localeResolver():

1
2
3
4
5
6
@Bean
public LocaleResolver localeResolver() {
    CookieLocaleResolver cookieLocaleResolver = new CookieLocaleResolver();
    cookieLocaleResolver.setDefaultLocale(Locale.ENGLISH);
    return cookieLocaleResolver;
}
This locale resolver is a CookieLocaleResolver which means that it stores the locale information in a client side cookie; as such – it will remember the user’s locale every time they log in, and during the entire visit.
Alternatively there is a SessionLocaleResolver, which remembers the locale throughout the session. To use this LocaleResolver instead, we need to replace the above method with the following:

1
2
3
4
5
@Bean
public LocaleResolver localeResolver() {
    SessionLocaleResolver sessionLocaleResolver = new SessionLocaleResolver();
    return sessionLocaleResolver;
}
Lastly, note that the LocaleChangeInterceptor will change the locale based on the value of a lang parameter sent with the login page by simple links:

1
2
<a href="?lang=en">English</a> |
<a href="?lang=es_ES">Spanish</a>

3.3. JSP/JSLT Localization

JSP/JSLT API will be used to display localized messages that are caught in the jsp page itself. To use the jsp localization libraries we should add the following dependencies to the pom.xml:

1
2
3
4
5
6
7
8
9
10
<dependency>
    <groupId>javax.servlet.jsp</groupId>
    <artifactId>javax.servlet.jsp-api</artifactId>
    <version>2.3.2-b01</version>
</dependency>
<dependency>
    <groupId>javax.servlet</groupId>
    <artifactId>jstl</artifactId>
    <version>1.2</version>
</dependency>

4. Displaying Error Messages

4.1. Login Validation Errors

In order to use the JSP/JSTL support and display localized messages in the login.jsp lets implement the following changes in the page:
1. Add the following tag lib element to the login.jsp:

1
<%@ taglib uri="http://java.sun.com/jsp/jstl/fmt" prefix="fmt"%>
2. Add the jsp/jslt element that will point to the messages.properties files:

1
<fmt:setBundle basename="messages" />
3. Add the following <fmt:…> elements to store the messages on jsp variables:

1
2
<fmt:message key="message.password" var="noPass" />
<fmt:message key="message.username" var="noUser" />
4. Modify the login validation script we saw in Section 3 so as to localize the error messages:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
<script type="text/javascript">
function validate() {
    if (document.f.j_username.value == "" && document.f.j_password.value == "") {
        alert("${noUser} and ${noPass}");
   document.f.j_username.focus();
   return false;
    }
    if (document.f.j_username.value == "") {
   alert("${noUser}");
   document.f.j_username.focus();
   return false;
     }
     if (document.f.j_password.value == "") {
   alert("${noPass}");
   document.f.j_password.focus();
   return false;
     }
}
</script>

4.2. Pre Login Errors

Sometimes the login page will be passed an error parameter if the previous operation failed. For example, a registration form submit button will load the login page. If the registration was successful, then it would be a good idea to show a success message in the login form, and an error message if the opposite was true.
In the sample login form below, we are implementing this by intercepting and regSucc and regError parameters, and displaying a localized message based on their values.

1
2
3
4
5
6
7
8
9
10
11
12
<c:if test="${param.regSucc == true}">
    <div id="status">
   <spring:message code="message.regSucc">   
        </spring:message>
    </div>
</c:if>
<c:if test="${param.regError == true}">
    <div id="error">
        <spring:message code="message.regError">  
        </spring:message>
    </div>
</c:if>

4.3. Login Security Errors

In case the login process fails for some reason, Spring Security will do a redirect to a login error URL, which we have defined to be /login.html?error=true.
So – similarly to how we have shown the status of the registration in the page, we need to do the same in case of a login problem:

1
2
3
4
5
6
<c:if test="${param.error != null}">
    <div id="error">
        <spring:message code="message.badCredentials">  
        </spring:message>
    </div>
</c:if>
Notice that we are using a <spring:message …> element. This means that the error messages are generated during the Spring MVC processing.
The full login page – including the js validation and these additional status messages can be found in the github project.

4.4. Logout Errors

In the example that follows, the jsp code <c:if test=”${not empty SPRING_SECURITY_LAST_EXCEPTION}”> in the logout.html page will check if there was an error in the logout process.
For example – if there is a persistence exception when a custom logout handler tries to store user data before redirecting to the logout page. While these errors are rare, we should handle them as neatly as possible as well.
Lets take a look at the complete logout.jsp:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
<%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core"%>
<%@ taglib prefix="sec"
    uri="http://www.springframework.org/security/tags"%>
<%@taglib uri="http://www.springframework.org/tags" prefix="spring"%>
<c:if test="${not empty SPRING_SECURITY_LAST_EXCEPTION}">
    <div id="error">
        <spring:message code="message.logoutError">   
        </spring:message>
    </div>
</c:if>
<c:if test="${param.logSucc == true}">
    <div id="success">
   <spring:message code="message.logoutSucc">   
        </spring:message>
    </div>
</c:if>
<html>
<head>
<title>Logged Out</title>
</head>
<body>  
    <a href="login.html">Login</a>
</body>
</html>
Notice that the logout page also reads the query string param logSucc, and if its value equal to true, a localized success message will be displayed.

5. The Spring Security Configuration

The focus of this article is the frontend of the login process, not the backend – so we’ll look only briefly at the main points of the security configuration;

5.1. Redirecting to the Login Error URL

The following directive in the <form-login…/> element directs the flow of the application to the url where the login error will be handled:

1
authentication-failure-url="/login.html?error=true"

5.2. The Logout Success Redirect


1
2
<logout invalidate-session="false" logout-success-url="/logout.html?logSucc=true"
    logout-url="/j_spring_security_logout" delete-cookies="JSESSIONID" />
The logout-success-url attribute simply redirects to the logout page with a parameter that confirms that the logout was successful.

6. Conclusion

In this article I have illustrated how to implement a Login page for a Spring Security backed application – handling login validation, displaying authentication errors and message localization.
We’re going to be looking at a full registration implementation in the next article – with the goal of having a full implementation of the login and registration process ready for production.
Posted by at
Labels: ,

Sunday, October 26, 2014

The @Scheduled Annotation in Spring

1. Overview

In this article we’ll discuss the Spring @Scheduled annotation – we will illustrate how it can be used to configure and schedule tasks.
The simple rules that need to be followed to annotate a method with @Scheduled are:
  • method should have void return type
  • method should not accept any parameters

2. Enable Support for Scheduling

To enable the support for scheduling tasks and the @Scheduled annotation in Spring – we can use the Java enable-style annotation:

1
2
3
4
5
@Configuration
@EnableScheduling
public class SpringConfig {
    ...
}
Or we can do the same in XML:

1
<task:annotation-driven>

3. Schedule Task at Fixed Delay

Let’s start by configuring a task to run after a fixed delay:

1
2
3
4
@Scheduled(fixedDelay = 1000)
public void scheduleFixedDelayTask() {
    System.out.println("Fixed delay task - " + System.currentTimeMillis()/1000);
}
In this case, the duration between the end of last execution and the start of next execution is fixed. The task always waits until the previous one is finished.
This option should be used when it’s mandatory that the previous execution is completed before running again.

4. Schedule Task at a Fixed Rate

Let’s not execute a task at a fixed interval of time:

1
2
3
4
@Scheduled(fixedRate = 1000)
public void scheduleFixedRateTask() {
    System.out.println("Fixed rate task - " + System.currentTimeMillis()/1000);
}
Note that the beginning of the task execution doesn’t wait for the completion of the previous execution.
This option should be used when each execution of the task is independent.

5. Schedule Task with Initial Delay

Next – let’s schedule a task with a delay (in milliseconds):

1
2
3
4
5
@Scheduled(fixedDelay = 1000, initialDelay = 1000)
public void scheduleFixedRateWithInitialDelayTask() {
    long now = System.currentTimeMillis() / 1000;
    System.out.println("Fixed rate task with one second initial delay - " + now);
}
Note how we’re using both fixedDelay as well as initialDelay in this example. The task will be executed a first time after the initialDelay value – and it will continue to be executed according to the fixedDelay.
This option comes handy when the task has set-up that need to be completed.

6. Schedule Task using Cron Expressions

Sometimes delays and rates are not enough, and we need the flexibility of a cron expression to control the schedule of our tasks:

1
2
3
4
5
@Scheduled(cron = "0 15 10 15 * ?")
public void scheduleTaskUsingCronExpression() {
    long now = System.currentTimeMillis() / 1000;
    System.out.println("schedule tasks using cron jobs - " + now);
}
Note – in this example, that we’re scheduling a task to be executed at 10:15 AM on the 15th day of every month.

7. Parameterizing the Schedule

Hardcoding these schedules is simple, but usually you need to be able to control the schedule without re-compiling and re-deploying the entire app.
We’ll make use of Spring Expressions to externalize the configuration of the tasks – and we’ll store these in properties files:
A fixedDelay task:

1
@Scheduled(fixedDelayString = "${fixedDelay.in.milliseconds}")
A fixedRate task:

1
@Scheduled(fixedRateString = "${fixedRate.in.milliseconds}")
A cron expression based task:

1
@Scheduled(cron = "${cron.expression}")

8. Configuring scheduled tasks using XML

Spring also provides XML way of configuring the scheduled tasks – here is the XML configuration to set these up:

1
2
3
4
5
6
7
8
9
<!-- Configure the scheduler -->
<task:scheduler id="myScheduler" pool-size="10" />
 
<!-- Configure the fixedDealy, fixedRate or cron based schduled tasks -->
<task:scheduled-tasks scheduler="myScheduler">
    <task:scheduled ref="beanA" method="methodA" fixed-delay="5000" initial-delay="1000" />
    <task:scheduled ref="beanB" method="methodB" fixed-rate="5000" />
    <task:scheduled ref="beanC" method="methodC" cron="*/5 * * * * MON-FRI" />
</task:scheduled-tasks>

9. Conclusion

In this article we understood the way to configure and use @Scheduled annotation. We covered the process to enable scheduling and various ways to configure scheduling task pattern
Posted by at
Labels: ,

Thursday, October 23, 2014

Java Network Programming Questions

Networking and Socket Programming is one of the important area of Java programming language, especially for those programmers, who are working in client server based applications. Knowledge of important protocols e.g. TCP and UDP in detail is very important, especially if you are in business of writing high frequency trading application, which communicate via FIX Protocol or native exchange protocol. In this article, we will some of the frequently asked questions on networking and socket programming, mostly based around TCP IP protocol. This article is kinda light on NIO though, as it doesn't include questions from multiplexing, selectors, ByteBuffer and FileChannel but it does include classical questions like difference between IO and NIO. Main focus of this post is to make Java developer familiar with low level parts e.g. how TCP and UDP protocol works, socket options and writing multi-threaded servers in Java. Questions discussed here is not really tied up with Java programming language, and can be used in any programming language, which allows programmers to write client-server applications. By the way, If you are going for interview on Investment banks for core Java developer role, you better prepare well on Java NIO, Socket Programming, TCP, UDP and Networking along with other popular topics e.g. multi-threading, Collections API and Garbage Collection tuning. You can also contribute any question, which is asked to you or related to socket programming and networking and can be useful for Java interviews.




Java Networking and Socket Programming Questions Answers

Here is my list of 15 interview questions related to networking basics, internet protocol and socket programming in Java. Though it doesn't contain basic questions form API e.g. Server, ServerSocket, but it focus on high level concept of writing scalable server in Java using NIO selectors and how to implement that using threads, there limitations and issues etc. I will probably add few more questions based on some best practices while writing socket based application in Java. If you know a good question on this topic, feel free to suggest.


1) Difference between TCP and UDP protocol?
There are many differences between TCP (Transmission control Protocol) and UDP (User Datagram Protocol), but main is TCP is connection oriented, while UDP is connection less. This means TCP provides guaranteed delivery of messages in the order they are sent, while UDP doesn't provide any delivery guarantee. Because of this guarantee, TCP is slower than UDP, as it needs to perform more work. TCP is best suited for message, which you can't afford to loss, e.g. order and trade messages in electronic trading, wire transfer in banking and finance etc. UDP is more suited for media transmission, where loss of one packet, known as datagrams is affordable and doesn't affect quality of service. This answer is enough for most of the interviews, but you need to be more detailed when you are interviewing as Java developer for high frequency trading desk. Some of the points which many candidate forget to mention is about order and data boundary. In TCP, messages are guaranteed to be delivered in the same order as they are sent but data boundary is not preserved, which means multiple messages can be combined and sent together, or receiver may receive one part of the message in one packet and other part of the message in next packet. Though application will receive full message and in the same order. TCP protocol will do assembling of message for you. On the other hand, UDP sends full message in a datagram packet, if clients receives the packet it is guaranteed that it will get the full message, but there is no guarantee that packet will come in same order they are sent. In short, you must mention following differences between TCP and UDP protocol while answering during interview :

  • TCP is guaranteed delivery, UDP is not guaranteed.
  • TCP guarantees order of messages, UDP doesn't.
  • Data boundary is not preserved in TCP, but UDP preserves it.
  • TCP is slower compared to UDP.


2) How does TCP handshake works?
Three messages are exchanged as part of TCP head-shake e.g. Initiator sends SYN,  upon receiving this Listener sends SYN-ACK, and finally initiator replied with ACK, at this point TCP connection is moved to ESTABLISHED state. This process is easily understandable by looking at following diagram.

Java Networking Interview Questions and Answers




3) How do you implement reliable transmission in UDP protocol?
This is usually follow-up of previous interview question. Though UDP doesn't provide delivery guarantee at protocol level, you can introduce your own logic to maintain reliable messaging e.g. by introducing sequence numbers and retransmission. If receiver find that it has missed a sequence number, it can ask for replay of that message from Server. TRDP protocol, which is used Tibco Rendezvous (a popular high speed messaging middle-ware) uses UDP for faster messaging and provides reliability guarantee by using sequence number and retransmission.


4) What is Network Byte Order? How does two host communicate if they have different byte-ordering?
There are two ways to store two bytes in memory, little endian (least significant byte at the starting address) and big endian (most significant byte at the starting address). They are collectively known as host byte order. For example, an Intel processor stores the 32-bit integer as four consecutive bytes in memory in the order 1-2-3-4, where 1 is the most significant byte. IBM PowerPC processors would store the integer in the byte order 4-3-2-1. Networking protocols such as TCP are based on a specific network byte order, which uses big-endian byte ordering. If two machines are communicating with each other and they have different byte ordering, they are converted to network byte order before sending or after receiving. Therefore, a little endian micro-controller sending to a UDP/IP network must swap the order in which bytes appear within multi byte values before the values are sent onto the network, and must swap the order in which bytes appear in multi byte values received from the network before the values are used. In short, you can also say network byte order is standard of storing byte during transmission, and it uses big endian byte ordering mechanism.


5) What is Nagle's algorithm?
If interviewer is testing your knowledge of TCP/IP protocol than it's very rare for him not to ask this question. Nagle's algorithm is way of improving performance of TCP/IP protocol and networks by reducing number of TCP packets that needs to be sent over network. It works by buffering small packets until buffer reaches Maximum Segment Size. Since small packets, which contains only 1 or 2 bytes of data, has more overhead in terms of TCP header, which is of 40 bytes. These small packets can also leads to congestion in slow network. Nagle's algorithm tries to improve efficiency of TCP protocol by buffering them, to send a larger packet. Also Nagle's algorithm has negative effect on non small writes, so if you are writing large data on packets than it's better to disable Nagle's algorithm. In general, Nagle's algorithm is a defence against careless application, which sends lots of small packets to network, but it will not benefit or have a negative effect on well written application, which properly takes care of buffering.


6) What is TCP_NODELAY?
TCP_NODELAY is an option to disable Nagle's algorithm, provided by various TCP implementations. Since Nagle's algorithm performs badly with TCP delayed acknowledgement algorithm, it's better to disable Nagle's when you are doing write-write-read operation. Where a read after two successive write on socket may get delayed up-to 500 millisecond, until the second write has reached the destination. If latency is more concern over bandwidth usage e.g. in a network based multi-player game, user wants to see action from other player immediately, it's better to bypass Nagle's delay by using TCP_NODELAY flag.


7) What is multicasting or multicast transmission? Which Protocol is generally used for multicast? TCP or UDP?
Multi-casting or multicast transmission is one to many distribution, where message is delivered to a group of subscribers simultaneously in a single transmission from publisher. Copies of messages are automatically created in other network elements e.g. Routers, but only when the topology of network requires it. Tibco Rendezvous supports multicast transmission. Multi-casting can only be implemented using UDP, because it sends full data as datagram package, which can be replicated and delivered to other subscribers. Since TCP is a point-to-point protocol, it can not deliver messages to multiple subscriber, until it has link between each of them. Though, UDP is not reliable, and messages may be lost or delivered out of order. Reliable multicast protocols such as Pragmatic General Multicast (PGM) have been developed to add loss detection and retransmission on top of IP multicast. IP multicast is widely deployed in enterprises, commercial stock exchanges, and multimedia content delivery networks. A common enterprise use of IP multicast is for IPTV applications


8) What is difference between Topic and Queue in JMS?
Main difference between Topic and Queue in Java Messaging Service comes when we have multiple consumers to consumer messages. If we set-up multiple listener thread to consume messages from Queue, each messages will be dispatched to only one thread and not all thread. On the other hand in case of Topic each subscriber gets it's own copy of message.


9) What is difference between IO and NIO?
Main difference between NIO and IO is that NIO provides asynchronous, non blocking IO, which is critical to write faster and scalable networking systems. While most of utility from IO classes are blocking and slow. NIO take advantage of asynchronous system calls in UNIX systems such as select() system call for network sockets. Using select(), an application can monitor several resources at the same time and can also poll for network activity without blocking. The select() system call identifies if data is pending or not, then read() or write() may be used knowing that they will complete immediately.


10) How do you write multi-threaded server in Java?
A multi-threaded server is the one which can server multiple clients without blocking. Java provides excellent support to developer such server. Prior to Java 1.4,  you can write multi-threaded server using traditional socket IO and threads. This had severe limitation on scalability, because  it creates new thread for each connection and you can only create a fixed number of threads, depending upon machine's and platform's capability. Though this design can be improved by using thread pools and worker threads, it still a resource intensive design. After JDK 1.4 and NIO's introduction, writing scalable and multi-threaded server become bit easier. You can easily create it in single thread by using Selector, which takes advantage of asynchronous and non-blocking IO model of Java NIO.


11) What is ephemeral port?
In TCP/IP connection usually contains four things, Server IP, Server port, Client IP and Client Port. Out of these four, 3 are well known in most of the time, what is not known is client port, this is where ephemeral ports comes into picture. ephemeral ports are dynamic port assigned by your machine's IP stack, from a specified range, known as ephemeral port range, when a client connection explicitly doesn't specify a port number. These are short lived, temporary port, which can be reused once connection is closed, but most of IP software, doesn't reuse ephemeral port, until whole range is exhausted. Similar to TCP, UDP protocol also uses ephemeral port, while sending datagram . In Linux ephemeral port range is from 32768 to 61000, while in windows default ephemeral port range is 1025 to 5000. Similarly different operating system has different ephemeral port ranges


12) What is sliding window protocol?
Sliding window protocol is a technique for controlling transmitted data packets between two network computers where reliable and sequential delivery of data packets is required, such as provided by Transmission Control Protocol (TCP). In the sliding window technique, each packet includes a unique consecutive sequence number, which is used by the receiving computer to place data in the correct order. The objective of the sliding window technique is to use the sequence numbers to avoid duplicate data and to request missing data


13) When do you get "too many files open" error?
Just like File connection, Socket Connection also needs file descriptors, Since every machine has limited number of file descriptors, it's possible that they may ran out of file descriptors. When it happen, you will see "too many files open" error. You can check how many file descriptor per process is allowed on UNIX based system by executing ulimit -n command or simply count entries on /proc//fd/


14) What is TIME_WAIT state in TCP protocol? When does a socket connection goes to TIME_WAIT state?
When one end of TCP Connection closes it by making system call, it goes into TIME_WAIT state. Since TCP packets can arrive in wrong order, the port must not be closed immediately to allow late packets to arrive. That's why that end of TCP connection goes into TIME_WAIT state. For example, if client closes a socket connection than it will go to TIME_WAIT state, similarly if server closes connection than you will see TIME_WAIT there. You can check status of your TCP and UDP sockets by using these networking commands in UNIX.


15) What will happen if you have too many socket connections in TIME_WAIT state on Server?
When a socket connection or port goes into TIME_WAIT state, it doesn't release file descriptor associated with it. File descriptor is only released when TIME_WAIT state is gone i.e. after some specified configured time. If too many connections are in TIME_WAIT state than your Server may ran out of file descriptors and start throwing "too many files open" error, and stop accepting new connections.


That's all about in this list of networking and socket programming interview questions and answers. Though I have originally intended this list for Java programmers it is equally useful for any programmer. In fact, this is bare minimum knowledge of sockets and protocols every programmer should have. I have found that C and C++ programmers are better answering these questions than an average Java programmer. One reason of this may be because Java programmers has got so many useful library e.g. Apache MINA, which does all the low level work for them. Anyway, knowledge of fundamentals is very important and everything else is just an excuse, but at same point I also recommend using tried and tested libraries like Apache MINA for production code.
Posted by at
Labels: ,

Out of the box JDK tools for troubleshooting

Not that you asked me, but I believe that software engineering is mostly about solving problems. All those blog posts saying “every child should learn to code” and “pff, math? I’m a programmer and I’m doing so much more than math, have you seen my HTML5 debugger thingy” agree that software engineering is more than writing pieces of prose in specific languages that computers know to understand. Problems our software solves defines us as programmers.
The ultimate advancement for how we tackle problems comes from science, an enhanced, clear mindset and the tools we utilize along the way.
jdk-tools list
Have you ever looked at what tools are bundled with your JDK installation? There ought to be something useful in there, right, since someone allegedly smart agreed to bundle it in ;-)
So in this post I offer a filtered list of utilities that are available in a typical installation of HotSpot Java. We decided to omit security related tools, tools for various RMI, applets, web-start and web-services. Let’s focus on what can be useful during development of normal apps by normal devs.
Now, as we mentioned above, this is not a full list, but we want to give you even better distilled version. Here are examples of what actually useful things can you do with these commands.

0. javap

The most useful variation of flags that you can pass to the javap – the Java Class Disassembler is the following:
  • -l – print line and local variable
  • -p – print information about non-public items as well
  • -c – print method bytecode
For example in the famous “Do you really get Classloaders?” presentation, when an NoSuchMethodException occurs, we investigate what members the class actually has, we execute:
javap -l -c -p Util2
and get all the info about the class we’re looking for.
javap example
For debugging what do your classes look like or just for gazing at random bytecode sequences, javap is a great tool to have.

1. jjs

You get a JavaScript console that you can use as a calculator or to test the weirdness of JS by executing random JS strings. Never let another JavaScript puzzle catch you unprepared!
jjs example
Huh, did you see that coming? But JavaScript is a topic for another time, just know that with jjs you can always check how it works, even if you don’t have node.js or a browser at hand.

2. jhat

The Java Heap Analysis Tool (jhat) and is used exactly how it’s described: analysing heap dumps. In our small example, we’ll generate that by causing an OutOfMemoryError and supplying a -XX:+HeapDumpOnOutOfMemoryError option of the Java process, so it will produce a file for us to analyze.
Causing an OutOfMemoryError is easy (most of the time we do it without wanting to!), we’ll just continuously generate garbage without allowing the garbage collector to intervene:
public class OhMyMemory {
 
 private static Map<String, Integer> map = new HashMap<>();
 
 public static void main(String[] args) {
   Runtime.getRuntime().addShutdownHook(
     new Thread() {
       @Override
       public void run() {
         System.out.println("We have accumulated " + map.size() + " entries");
       }
     }
   );
   for(int i = 0; ;i++) {
     map.put(Integer.toBinaryString(i), i);
   }
 }
}
When we run this snippet, it produces the following output:
org.shelajev.throwaway.jdktools.OhMyMemory
java.lang.OutOfMemoryError: Java heap space
Dumping heap to java_pid5644.hprof ...
Heap dump file created [73169721 bytes in 0.645 secs]
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.HashMap.resize(HashMap.java:703)
at java.util.HashMap.putVal(HashMap.java:662)
at java.util.HashMap.put(HashMap.java:611)
at org.shelajev.throwaway.jdktools.OhMyMemory.main(OhMyMemory.java:24)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
We have accumulated 393217 entries
Great success, we have a file now. Let’s aim jhat at it and start the analysis. It crunches the file and spins off a http server for us to get the results.
shelajev@shrimp  ~/workspace_idea/throwaway
$ jhat java_pid5644.hprof
Reading from java_pid5644.hprof...
Dump file created Thu Aug 14 14:48:19 EEST 2014
Snapshot read, resolving...
Resolving 1581103 objects...
Chasing references, expect 316 dots...
Eliminating duplicate references........
Snapshot resolved.
Started HTTP server on port 7000
Server is ready.
Visiting http://localhost:7000 reveals the webpage with links to the data.
jhat all classes view
And from there we can look at, for example, the Heap Histogram to figure out what is eating our memory.
jhat heap histogram
Now it’s clear that our HashMap with 393567 nodes is what crashes the program. There are more useful tools to check the layout of memory in use and analyze the heaps. However, jhat is built in, always available and is a good starting point for any analysis.

3. jmap

Jmap is a memory mapping tool that provides another way to obtain heap dumps without the need to cause any OutOfMemoryErrors. Let’s change the above program a bit to show that in action.
public class OhMyMemory {
 
 private static Map<String, Integer> map = new HashMap<>();
 
 public static void main(String[] args) {
   Runtime.getRuntime().addShutdownHook(
     new Thread() {
       @Override
       public void run() {
         try {
           System.out.println("Enter something, so I'll release the process");
           System.in.read();
           System.out.println("We have accumulated " + map.size() + " entries");
         }
         catch (IOException e) {
           e.printStackTrace();
         }
       }
     }
   );
 
   for(int i = 0; i < 10000 ;i++) {
     map.put(Integer.toBinaryString(i), i);
   }
 }
}
Note that now we don’t consume a humongous amount of memory, but finish early and just sit in the shutdown hook not allowing the JVM to exit. This allows us to connect to this program’s process with jmap and get the precious memory dump.
So there are two very useful things you can do with jmap, getting the summary of a heap and triggering a heap dump. So when I execute:
jmap -heap 1354, where 1354 is the id of the Java process running the code above, I get nice info about the memory usage:
shelajev@shrimp  ~/workspace_idea/throwaway
$ jmap -heap 1354                                                                                                                   
Attaching to process ID 1354, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 25.0-b70
 
using thread-local object allocation.
Parallel GC with 4 thread(s)
 
Heap Configuration:
   MinHeapFreeRatio         = 40
   MaxHeapFreeRatio         = 70
   MaxHeapSize              = 67108864 (64.0MB)
   NewSize                  = 1572864 (1.5MB)
   MaxNewSize               = 22020096 (21.0MB)
   OldSize                  = 45088768 (43.0MB)
   NewRatio                 = 2
   SurvivorRatio            = 8
   MetaspaceSize            = 21807104 (20.796875MB)
   CompressedClassSpaceSize = 1073741824 (1024.0MB)
   MaxMetaspaceSize         = 17592186044415 MB
   G1HeapRegionSize         = 0 (0.0MB)
 
Heap Usage:
PS Young Generation
Eden Space:
   capacity = 1048576 (1.0MB)
   used     = 628184 (0.5990829467773438MB)
   free     = 420392 (0.40091705322265625MB)
   59.908294677734375% used
From Space:
   capacity = 524288 (0.5MB)
   used     = 491568 (0.4687957763671875MB)
   free     = 32720 (0.0312042236328125MB)
   93.7591552734375% used
To Space:
   capacity = 524288 (0.5MB)
   used     = 0 (0.0MB)
   free     = 524288 (0.5MB)
   0.0% used
PS Old Generation
   capacity = 45088768 (43.0MB)
   used     = 884736 (0.84375MB)
   free     = 44204032 (42.15625MB)
   1.9622093023255813% used
 
981 interned Strings occupying 64824 bytes.
Also it might be easier just to trigger the dump and analyze that later at your leisure. To do that, pass the -dump flag to jmap, like in the example below:
shelajev@shrimp  ~/workspace_idea/throwaway
$ jmap -dump:live,format=b,file=heap.bin 1354                                                                               
Dumping heap to /Users/shelajev/workspace_idea/throwaway/heap.bin ...
Heap dump file created
Now you have the dump file heap.bin, which you can feed to a memory analyzer of your choosing.

4. jps

Jps is a process status tool that is most commonly used to determine process ID (PID) of the Java process. It works in an OS-independent way and is really convenient. Imagine that we have started the program above and want to connect to it with jmap. For that we need a PID and that is the exact scenario where jps comes to the rescue.
shelajev@shrimp  ~/workspace_idea/throwaway
$ jps -mlv
5911 com.intellij.rt.execution.application.AppMain org.shelajev.throwaway.jdktools.OhMyMemory -Xmx64m -Didea.launcher.port=7535 -Didea.launcher.bin.path=/Applications/IntelliJ IDEA 14 EAP.app/Contents/bin -Dfile.encoding=UTF-8
5544  -Dfile.encoding=UTF-8 -ea -Dsun.io.useCanonCaches=false -Djava.net.preferIPv4Stack=true -Djsse.enableSNIExtension=false -XX:+UseConcMarkSweepGC -XX:SoftRefLRUPolicyMSPerMB=50 -XX:+HeapDumpOnOutOfMemoryError -Xverify:none -Xbootclasspath/a:../lib/boot.jar -Xms128m -Xmx750m -XX:MaxPermSize=350m -XX:ReservedCodeCacheSize=225m -XX:+UseCompressedOops -agentlib:yjpagent=probe_disable=*,disablealloc,disabletracing,onlylocal,disableexceptiontelemetry,delay=10000,sessionname=IntelliJIdea14 -Didea.java.redist=NoJavaDistribution -Didea.home.path=/Applications/IntelliJ IDEA 14 EAP.app/Contents -Didea.paths.selector=IntelliJIdea14
5930 sun.tools.jps.Jps -mlvV -Dapplication.home=/Library/Java/JavaVirtualMachines/jdk1.8.0.jdk/Contents/Home -Xms8m
We find that the combination of flags “-mlv” works the best in most cases. It print out main method args, full package names and arguments given to the JVM. This makes quite easy to identify the process you need even among similar ones.

5. jstack

Jstack is a utility to produce threads’ stacktraces of a given JVM process. If you think there is a deadlock and just want to verify what your threads do while you’re shown that progress bar, jstack is for you.
There are just a couple of options that jstack can accept, so when you’re in doubt throw all of them in. Later you can always limit the output, when you can see what information is less useful.
Useful options include -F to force the dump, which can be used on hang processes, -l to print info about synchronisation and locks.
shelajev@shrimp  ~/workspace_idea/throwaway
$ jstack -F -l 9153
Attaching to process ID 9153, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 25.0-b70
Deadlock Detection:
 
No deadlocks found.
….
The output above is cut for brevity, but it originally contains information about in which state every thread is and what’s its current stack trace.
“Jstack is extremely useful, we are using that extensively, especially on our test engine that is responsible for starting / stopping the application servers we test with. Things often go wrong and it gives us the insight to what JVMs are doing when there are no visible side-effects.”
Posted by at
Labels:
Subscribe to: Posts (Atom)