tag:blogger.com,1999:blog-49577428415157871982024-03-11T03:13:18.686+08:00Tech TotsRandom thoughts on Linux, Java, security and other technical stuff...Mikehttp://www.blogger.com/profile/12828408134779227322noreply@blogger.comBlogger89125tag:blogger.com,1999:blog-4957742841515787198.post-60813181922668731692021-06-09T20:59:00.006+08:002021-06-13T15:44:27.772+08:00Enabling PAM Authentication in Hive (HDP 2.6) OverviewThis article describes how to enable PAM authentication in Hive. By default, there's no authentication to Hive server. With PAM, authentication is performed against local OS user credentials.ProcedureStep 1 - JPam LibraryDownload latest copy JPam library from http://jpam.sourceforge.net/Latest version is 1.1 (JPam-Linux_amd64-1.1.tgz)Once downloaded, create /usr/hdp/ext/Mikehttp://www.blogger.com/profile/12828408134779227322noreply@blogger.com2tag:blogger.com,1999:blog-4957742841515787198.post-48982676820749364802020-03-21T19:29:00.000+08:002020-03-21T19:29:02.933+08:00Clearing table locks in MariaDB ColumnStore (MCS)If you encounter the following error running MCS:
Internal error: CAL0009: Truncate table failed: IDB-2009: Unable to perform the cpimport operation because 30084 with PID -1 is currently holding the table lock for session .
It's due to locks placed on the objects. Use the following command to view locked objects in MCS:
$ /usr/local/mariadb/columnstore/bin/Mikehttp://www.blogger.com/profile/12828408134779227322noreply@blogger.com9tag:blogger.com,1999:blog-4957742841515787198.post-40582885734710480082017-03-09T19:07:00.001+08:002017-03-09T19:07:14.410+08:00Hive External TableScript to create an external table in Hive to read records from a HDFS folder:
CREATE EXTERNAL TABLE Mytable (
Id int,
PlanID string,
ServiceID string,
SessionDuration int)
STORED AS TEXTFILE
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ","
LOCATION "/user/hive/staging/"
TBLPROPERTIES(
"skip.header.line.count"="1"
);
Using the script above, it'll use the files in /user/hive/Mikehttp://www.blogger.com/profile/12828408134779227322noreply@blogger.com13tag:blogger.com,1999:blog-4957742841515787198.post-81828928596260861782016-08-23T14:09:00.001+08:002016-08-23T14:24:11.577+08:00Enabling client authentication in MongoDBBefore enabling authentication in MongoDB, we'll have to create a user and assign it a built-in role. We'll use the built-in "root" role that provides admin access to all databases.
I've done this in Ubuntu using MongoDB 3.2.
Issue the following commands to create a new user in the "admin" database:
use admin
db.createUser({
user: "admin",
pwd: "password",
roles: [
{ Mikehttp://www.blogger.com/profile/12828408134779227322noreply@blogger.com6tag:blogger.com,1999:blog-4957742841515787198.post-60749268206889146832016-08-07T00:00:00.003+08:002016-08-23T14:24:27.151+08:00Limiting grep outputHere's a quick way to limit output of the grep command.
Most of the time, we issue the following to find needle in the haystack:
grep needle file.txt
This prints out the matching pattern. If the output is way too long and we only need a section of it, we can use the extended grep (a.k.a. egrep) option.
Here's how to use regular expressions to instruct grep to only display 40 characters Mikehttp://www.blogger.com/profile/12828408134779227322noreply@blogger.com3tag:blogger.com,1999:blog-4957742841515787198.post-39425357403296637852016-07-27T10:52:00.000+08:002016-08-23T14:23:41.767+08:00Enabling WS-Security in Spring Boot using CXF, JAX-WS and JAXBIn my previous post, I've shown how to quickly create a WSDL/SOAP based web service. This post will build on top of that to include WS-Security. We'll be using simple username/password authentication.
Add the following artifact into your Spring Boot pom.xml:
org.apache.cxf
cxf-rt-ws-security
3.1.6
We create a callback handler class to perform our custom authentication. ThisMikehttp://www.blogger.com/profile/12828408134779227322noreply@blogger.com7tag:blogger.com,1999:blog-4957742841515787198.post-46064464682325325812016-07-22T21:21:00.000+08:002016-08-23T14:25:23.059+08:00Getting Caused by: com.sun.xml.bind.v2.runtime.IllegalAnnotationsException: 1 counts of IllegalAnnotationExceptions...My previous post shows how we can easily create a SOAP based web service using Spring Boot, CXF, JAX-WS and JAXB.
There's a small matter to note when naming functions. The following function naming works:
package com.techtots.services;
import javax.jws.WebMethod;
import javax.jws.WebParam;
import javax.jws.WebResult;
import javax.jws.WebService;
import javax.xml.bind.annotation.XmlElement;
Mikehttp://www.blogger.com/profile/12828408134779227322noreply@blogger.com4tag:blogger.com,1999:blog-4957742841515787198.post-77830813414696960672016-07-22T21:09:00.001+08:002016-08-23T14:25:38.297+08:00Creating WSDL/SOAP web services in Spring Boot using CXF, JAX-WS and JAXBHere's a quick way to use Spring Boot to expose web services via WSDL/SOAP using CXF, JAX-WS and JAXB.
Add the following artifacts into your Spring Boot pom.xml:
org.apache.cxf
cxf-rt-frontend-jaxws
3.1.6
org.apache.cxf
cxf-rt-transports-http
3.1.6
We start off by creating the request and response classes. These classes act as the value objects for the Mikehttp://www.blogger.com/profile/12828408134779227322noreply@blogger.com1tag:blogger.com,1999:blog-4957742841515787198.post-32233090344905247202014-07-26T14:39:00.000+08:002014-08-05T17:41:15.385+08:00Configuring Apache to return CORS headers for Drupal ServicesHere's what I did to configure Apache to return the proper CORS headers for my webapp consumption which is written in AngularJS:
Header always set Access-Control-Allow-Origin "*"
Header always set Access-Control-Allow-Methods "GET, POST, OPTIONS"
Header always set Access-Control-Allow-Headers "accept, content-type"
Header always set Access-Control-Allow-Credentials "true"
That can be set in theMikehttp://www.blogger.com/profile/12828408134779227322noreply@blogger.com7tag:blogger.com,1999:blog-4957742841515787198.post-11173059626623406332014-02-11T15:28:00.001+08:002014-02-11T15:29:16.796+08:00Resizing images in batch using mogrifyHere's a quick way to resize images via shell using mogrify. It's part of ImageMagick package.
mogrify -resize 60% *.jpg
This will resize all files ending with .jpg extension to 60% of their original size.
We're only scratching the surface of what mogrify can do. Have fun exploring :)Mikehttp://www.blogger.com/profile/12828408134779227322noreply@blogger.com0tag:blogger.com,1999:blog-4957742841515787198.post-17844560601092202952014-02-03T10:01:00.000+08:002014-02-03T10:01:15.580+08:00Default Parameter Values in PHP FunctionsHere's a neat way to set some default parameter values in a PHP function. This only works if a function param is passed in as associative array.
function create_profile($values) {
if (!is_array($values)) {
$values = array();
}
// Default key/values
$values += array(
'uid' => NULL,
'username' => '',
'mobile' => '99999999',
'location' => 'POINT(0 0)',
Mikehttp://www.blogger.com/profile/12828408134779227322noreply@blogger.com1tag:blogger.com,1999:blog-4957742841515787198.post-46974534393286359312014-01-28T21:59:00.001+08:002014-01-28T21:59:54.692+08:00Setting "placeholder" attribute in Drupal 7 form elementsHere's a quick way on how to use HTML5 placeholder attribute in Drupal forms. Example below is altering the login form in a custom theme:
function mytheme_form_alter(&$form, &$form_state, $form_id) {
if ($form_id == 'user_login') {
$form['name']['#attributes'] = array('placeholder' => $form['name']['#title']);
$form['pass']['#attributes'] = array('placeholder' => $form['pass']['#Mikehttp://www.blogger.com/profile/12828408134779227322noreply@blogger.com0tag:blogger.com,1999:blog-4957742841515787198.post-63273440247929784312013-09-11T20:03:00.000+08:002013-09-11T20:03:13.436+08:00Using JMeter's Regular Expression ExtractorHere's how to use JMeter's Regular Expression Extractor post-processor to extract the response from a HTTP Request sampler and use it in the subsequent request.
In the screen above, I'm extracting the value of the hidden form element named "t". If there's no match to the regular expression, the default value "NO TOKEN" will be returned.
To use the extracted value in subsequent HTTP Mikehttp://www.blogger.com/profile/12828408134779227322noreply@blogger.com0tag:blogger.com,1999:blog-4957742841515787198.post-8856479108457189492013-08-15T01:46:00.000+08:002013-08-15T01:46:09.575+08:00Using Drupal's hook_theme() for custom themingThis is a short post on how to use Drupal's hook_theme() to customize a layout within your own custom module.
Start off with implementing the hook_theme() function in your module:
/**
* Implements hook_theme()
* @param type $existing
* @param type $type
* @param type $theme
* @param type $path
* @return array
*/
function mypm_theme($existing, $type, $theme, $path) {
$themes = array(
Mikehttp://www.blogger.com/profile/12828408134779227322noreply@blogger.com0tag:blogger.com,1999:blog-4957742841515787198.post-5380475805786669602013-08-14T21:17:00.000+08:002013-08-14T21:25:09.276+08:00Spring 3 MVC + Hibernate 4 + JdbcTemplate + Partial REST controller (Part 3)Last part of the article is to use either Hibernate or Spring's JdbcTemplate to retrieve data. The following class is the BaseDao where DataSource and Hibernate SessionFactory are injected and used by concrete DAO classes:
package com.techtots.services.api.dao;
import javax.sql.DataSource;
import org.hibernate.SessionFactory;
import org.springframework.beans.factory.annotation.Autowired;
Mikehttp://www.blogger.com/profile/12828408134779227322noreply@blogger.com0tag:blogger.com,1999:blog-4957742841515787198.post-55011657710593747842013-08-14T20:56:00.004+08:002013-08-14T21:18:01.652+08:00Spring 3 MVC + Hibernate 4 + JdbcTemplate + Partial REST controller (Part 2)Second part of the article will focus on the controllers which will process clients' requests as well as other helper classes.
We start off with a custom exception. We add on a HttpStatus code into the exception to return to clients:
package com.techtots.services.api.common.exceptions;
import org.springframework.http.HttpStatus;
public class RestException extends Exception {
private staticMikehttp://www.blogger.com/profile/12828408134779227322noreply@blogger.com0tag:blogger.com,1999:blog-4957742841515787198.post-87358150625629505992013-08-14T19:36:00.000+08:002013-08-14T19:36:44.560+08:00Spring 3 MVC + Hibernate 4 + JdbcTemplate + Partial REST controller (Part 1)This is post serves as a reference point to quickly setup a Java web application with Spring 3 MVC, Hibernate 4, JdbcTemplate and a partial REST controller. We'll be consuming and responding JSON content via the REST controllers.
I'm using Eclipse as the IDE and will be connecting to MySQL database.
Let's start off with the libraries:
antlr-2.7.7.jar
aopalliance.jar
bonecp-0.7.1.RELEASE.jar
Mikehttp://www.blogger.com/profile/12828408134779227322noreply@blogger.com1tag:blogger.com,1999:blog-4957742841515787198.post-7297060370153550502013-08-01T20:47:00.000+08:002013-08-01T20:47:12.606+08:00Two things to get Kohana up and runningHere are 2 things to do right after meeting all requirements dictated by Kohana's install script:
Set Cookie::$salt value in application/bootstrap.php:
Cookie::$salt = 'my*salt*value*';
Change base_url value in application/bootstrap.php to match the server configuration:
Kohana::init(array(
'base_url' => '/my_app/',
));
In this example, you should be calling http://myserver.com/my_app/
Mikehttp://www.blogger.com/profile/12828408134779227322noreply@blogger.com0tag:blogger.com,1999:blog-4957742841515787198.post-32928324190985226922013-07-31T12:07:00.000+08:002013-07-31T12:07:43.950+08:00Java Message Format Using Named PlaceholderThe Java MessageFormat class allows user to pre-define a string with placeholders and then fill the placeholders with actual strings later to construct a proper message.
It's all fine if you're used to numbered placeholders e.g. {0} and {1}. Since I'm used to Drupal's format_string() function, here's a better alternative. Apache Commons has a StrSubstitutor class which allows use of named Mikehttp://www.blogger.com/profile/12828408134779227322noreply@blogger.com4tag:blogger.com,1999:blog-4957742841515787198.post-50822757801785431842013-07-30T21:53:00.001+08:002013-07-30T21:53:24.238+08:00SQL UNION using Drupal db_selectHere's how to use Drupal's db_select function to perform a SQL UNION like below:
select uid from users where type = 'i'
union
select uid from users_archive where type = 'i'
Bad example, but I can't think of a proper example now :)
Drupal db_select version:
$q1 = db_select('users', 'u')
->fields('u', array('uid'))
->condition('type', 'i');
$q2 = db_select('users_archive', 'ua')Mikehttp://www.blogger.com/profile/12828408134779227322noreply@blogger.com0tag:blogger.com,1999:blog-4957742841515787198.post-7218710905799360032013-07-26T10:31:00.001+08:002013-07-26T10:31:56.826+08:00Getting last run SQL from CodeIgniter's DB classHere's a convenient way to obtain the last executed SQL query when using CodeIgniter's database class.
Code snippet for this example:
$this->db->select('status');
$this->db->from('tbl_user');
$this->db->where('user_id', $user_id);
$results = $this->db->get();
echo $this->db->last_query();
The last call to last_query() will output the actual SQL executed.Mikehttp://www.blogger.com/profile/12828408134779227322noreply@blogger.com0tag:blogger.com,1999:blog-4957742841515787198.post-13861247261166674112013-07-17T08:59:00.000+08:002013-07-17T08:59:35.101+08:00SQL JOINS using Drupal db_selectHere's how to use Drupal's db_select function to perform a SQL INNER JOIN like below:
SELECT n.*, u.name FROM node n
INNER JOIN users u on n.uid = u.uid
WHERE u.id = 3
Drupal db_select version:
$q = db_select('node', 'n');
$q->join('users', 'u', 'u.uid = n.uid');
$q->fields('n');
$q->fields('u', array('name'));
$q->condition('u.uid', 3);
$results = $q->execute();
Mikehttp://www.blogger.com/profile/12828408134779227322noreply@blogger.com0tag:blogger.com,1999:blog-4957742841515787198.post-44944419330424985192013-07-15T21:40:00.001+08:002013-07-15T21:40:33.524+08:00Output XLS file to servlet output stream using JExcelAPIHere's a quick way (although not the best one) to stream an Excel file generated by JExcelAPI (http://jexcelapi.sourceforge.net/). I'm using the servlet output stream in this case.
HttpServletResponse response = getContext().getResponse();
ServletOutputStream sos = response.getOutputStream();
WritableWorkbook workbook = Workbook.createWorkbook(sos);
WritableSheet sheet = Mikehttp://www.blogger.com/profile/12828408134779227322noreply@blogger.com0tag:blogger.com,1999:blog-4957742841515787198.post-42282094807312569902013-07-01T21:39:00.001+08:002013-07-17T19:20:13.563+08:00Formatting Drupal's UNIX timestamp datesDrupal stores date/time value as int columns in MySQL. Its value is UNIX timestamp based. You will not be able to determine the actual date/time by selecting from the table.
Here's a convenient way to convert the date/time columns directly from SQL:
SELECT cid, data, FROM_UNIXTIME(created) FROM main_cache
You can also use this in the WHERE clause like below:
SELECT COUNT( * )
FROM Mikehttp://www.blogger.com/profile/12828408134779227322noreply@blogger.com1tag:blogger.com,1999:blog-4957742841515787198.post-39133478529543033002013-06-14T23:00:00.000+08:002013-06-14T23:01:12.061+08:00Programmatically parse webform submissionsHere's how to retrieve a webform submission and process its submission values programmatically. We'll need two IDs. First is the webform node id (nid) and the second is the submission id (sid). You can retrieve these IDs from viewing the webform results listing. The URL is something like below:
http://localhost/dt/node/124/submission/18
In this case, 124 is the node id while 18 is the Mikehttp://www.blogger.com/profile/12828408134779227322noreply@blogger.com0