hive、hadoop面试题

全栈程序员-站长 • 2022年6月17日下午1:36 • 未分类 • 阅读 33

大家好，又见面了，我是你们的朋友全栈君。

有如下hive记录表records，记录车辆的过车信息：
create table records(
id string, //记录编号
indate string, //过车记录时间
plate_no string, //车辆号牌
device_id int, //经过的设备编号
)
partitioned by(month string,day string)
row format delimited fields terminated by ‘\t’ stored as ORC;

1请使用HQL得到最近一个月内晚上(晚22点-早6点)出现记录最多的车辆号牌Top10及次数
select
plate_no,
rank() over(sort by total desc) rk,
total
from
(
   select
   plate_no,
   count(1) total
   from
    (
     select
     indate,
     plate_no
     from
     records
     where
     From_unixtime(unix_timestamp(indate,”yyyy-MM-dd HH:mm:ss”),”yyyy-MM-dd”)>=date_sub(‘2020-04-26’,30)
     and
     From_unixtime(unix_timestamp(indate,”yyyy-MM-dd HH:mm:ss”),”yyyy-MM-dd”) <=’2020-04-26′
     and
     From_unixtime(unix_timestamp(indate,”yyyy-MM-dd HH:mm:ss”),”HH”) <=22
     and
     From_unixtime(unix_timestamp(indate,”yyyy-MM-dd HH:mm:ss”),”HH”) >=6
    ) t1
   group by plate_no
) t2
where rk = 10

2请用spark RDD将上述表中indate、plate_no、device_id三个字段记录重复的数据只保留一条
object interview02 {

def main(args: Array[String]): Unit = {

val session: SparkSession = SparkSession.builder().master(“local[*]”).appName(“interview”).getOrCreate()
val rdd: RDD[Row] = session.read.csv(“records”).rdd

    val distinct = rdd.map(row => ((“indate”,”plate_no”,”device_id”),(“id”)))
      .groupByKey()
    distinct.distinct().foreach(println)
    session.stop()
   }

}

3有string A和string B，分别由4096个随机的0或1组成，样例为String A = “01010111001….110010”,String B=”10101110….100101″，
现有公式double C=(A和B逐位与的和)/(A中1的个数*B中1的个数)，请用自己熟悉的语言实现出满足此公式的方法。
public class Method {

public static void main(String[] args) {

 String A = "0101010101101010";
        String B = "1010101010101011";

        char[] chars1 = A.toCharArray();
        char[] chars2 = B.toCharArray();
        int sum;
        Double sum2 = 0.0;
        int c = 0;
        int d = 0;
        for (int i = 0; i < chars1.length; i++) {
            char a = chars1[i];

//            for (int j= 0;j< chars2.length;j++){
            char b = chars2[i];


            sum = Integer.parseInt(String.valueOf(a)) & Integer.parseInt(String.valueOf(b));
            /* System.out.println(sum); */
            sum2 += sum;
        }
        for (int j = 0; j < chars1.length; j++) {
            char a = chars1[j];

            char b = chars2[j];
            if (Integer.parseInt(String.valueOf(a)) == 1) {

                c += 1;


            }
            if (Integer.parseInt(String.valueOf(b)) == 1) {

                d += 1;


            }



        }
        Double C = sum2/(c*d);
        System.out.println(sum2);
        System.out.println(c);
        System.out.println(d);
        System.out.println(C);

}
}

发布者：全栈程序员-站长，转载请注明出处：https://javaforall.net/149946.html原文链接：https://javaforall.net