Posts

Programming with Apache Spark and Cassandra -draft

Putting the knowledge gained so far in this  and frequent questions that many may ask and what we have asked ourselves. 1.1          What is the need for using Spark ? Spark gives you horizontal scale ability in a programmer friendly way. 1.2          But what about other options ? There are other options as well. I have listed them below, which describes and highlights Sparks place in the architecture Type Level of Granularity Descritpion LoadBalancer (nginx,haproxy) Request Level (usually HTTP requests) Works well for Request-response type client server protocols. Works also well in context of microservices in application program side However to scale the processing insdie the application programs this is inadequate Task Managers (celery, other MQ based) Task Level Helps to scale processing in the application program.Takes care of Task handling. However the onus is on the developer
Post a Comment
Read more

Best practises - Selenium WebDriver/ Java

Intermittent failure in Selenium Test cases ? After clicking the drill down sometimes web-elements are not found causing all summary table test cases to fail. Common Root Causes 1)  Prefer Selection By.ID className  then By.cssSelectior and only if all else fails use By.Xpath Selection by CSS  By.CSSSelector  should be preferred over XPath as this is more stable as it is natively supported by browser . XPAth is an abstraction provided by Selenium and not as performant. If XPath is used make sure that you have hand written the XPath and it is performant and not generic in that it has to to brute force search through the entire DOM to find your element. Where ever possible , use By.ID, else By.CSSSelector and in case of no other option use XPath after proper testing. You can use FireFinder plugin for FireFox (first add FireBug) to test your CSS or Xpath (if there is no way you can select by CSS) For example this XPath to finding the drill down element  //*[@id='scTab
Post a Comment
Read more

Python Profiling - Some hints

No time to compose fully; Here are some links which helped me in CPU profiling Python Profilers http://www.vrplumber.com/programming/runsnakerun/  python -m cProfile -o profile_out ANR_4G_IRAT.py  pacman -S kdesdk-kcachegrind  http://thirld.com/blog/2014/11/30/visualizing-the-results-of-profiling-python-code/  runsnakerun http://wiki.wxpython.org/How%20to%20install%20wxPython  pstats dechipering cprofile output  https://pymotw.com/2/profile/  https://docs.python.org/2/library/profile.html  yappi https://code.google.com/archive/p/yappi/  pyprof2calltree -k -i myscript.cprof  https://julien.danjou.info/blog/2015/guide-to-python-profiling-cprofile-concrete-case-carbonara  performance tips python  https://wiki.python.org/moin/PythonSpeed/PerformanceTips#Profiling_Code
Post a Comment
Read more

Java Script Development Guidlines

Image
The aim of the page is to give practical and widely adopted industry best practises and guidelines for all phases of JavaScript development lifecycle.   All the necessary tools and frameworks like code analysis and test frameworks are already integrated to our CI system and is ready for use and is already in use by teams Part 1: Sensitizing with JavaScript language     Persons coming from other languages may not appreciate certain coding guidelines that is recommended and set in  SONAR JS Analysis and that is also checked by popular tools like JSHint and JSLint; Example the function below will return 'undefined' and not  'Hello World'.  function main() { return 'Hello, World!'; } main(); ->>return's 'undefined'  This is because in JavaScript semi columns are not mandatory and  JS will add semi colon automatically during code interpretation; So it will add a semicolon after return making the function return nothing; Such and o
Post a Comment
Read more

Long running Java process resource consumption monitoring , leak detection and GC tuning

For easily monitoring the JVM metrics there is no better tool than Java VisualVM or its older counterpart JConsole; These two tools comes with the Java JDK. So it is absolutely necessary that you copy a JDK version to some temp directory in which your server runs. {Make sure that you do not install the JDK / put the java executable in the path.} Before going in JVM monitoring, it is essential to understand a little about Java's memory model. Note - This article is written with  HotSpot JVM in mind,which is one of the most commonly used one ; implemented by OpenJDK and Oracle - formerly from Sun. This is not the only one , notable being JRockit and others ; and some details would change with a different JVM. When you are running a Java Process in a Server you allocate the Memory for the Java Virtual Machine. This memory is split to process heap { what you give withe the Xmx Xms command}where it is again subdivided on generational basis (Young, Old)  and the PermGen - whic
Post a Comment
Read more

JBoss AS 7.1.1 and HornetQ Clustering in two nodes

Fist follow https://docs.jboss.org/author/display/AS71/AS7+Cluster+Howto Check the  domain.xml configuration in the Master node  ( for profile - full-ha part) for the things marked yellow Code as well as the domain.xml configuration  and the ports to be open is in GitHub  https://github.com/alexcpn/jboss_hornetq_clustering (Tested with AS 7.1.1) Invocation http: // :8330/MroControllerREST/rest/sessionbean/startmro?scopename=werwerwe http: // :8330/MroControllerREST/rest/sessionbean/startmro?scopename=werwerwe Will create a message/messages and send to MROQueue ; The MROExecutorMDB will receive it and post a message to MROTopic which will be received by MROControllerMDB in both the master and slave nodes (Deploy MROController.ear and MROExecuotor.ear to only other-server-group (jboss-cli.sh , connect to master and deploy)
Post a Comment
Read more

Exposing REST interface from a Session Bean

Here is a working example  at  https://github.com/alexcpn/rest_in_sessionbean  Tested with JBOSS AS 7.1.1. Note that a war file is needed in the ear, with a web.xml . This is using JBOSS inbuilt RESTEasy. Idea from Adam Bein's  blog -  www.adam-bien.com/roller/abien/entry/ejb_3_1_and_rest
Post a Comment
Read more

Apache CXF and Spring

http://cxf.apache.org/docs/writing-a-service-with-spring.html http://cxf.apache.org/docs/jax-ws-configuration.html http://cxf.apache.org/docs/configuration.html And finally à http://stackoverflow.com/questions/689902/cxf-without-spring --> http://cxf.apache.org/docs/servlet-transport.html#ServletTransport-UsingtheservlettransportwithoutSpring   Follwing maven settings in   jroller http://www.jroller.com/gmazza/entry/web_service_tutorial Reading a little from Spring I understood that this is actually the applicationcontext.xml and cxf is somehow cooking its beans with Spring < bean id="cxf" class="org.apache.cxf.bus.spring.SpringBus" destroy-method="shutdown" / >             < bean id="org.apache.cxf.bus.spring.BusWiringBeanFactoryPostProcessor"                 class="org.apache.cxf.bus.spring.BusWiringBeanFactoryPostProcessor" / >     < bean id="org.apache.cxf.bus.spring.Jsr25
Post a Comment
Read more